Vihang Patil

Transformers can be slow for real-time applications like robotics. We study if modern recurrent architectures, like xLSTM and Mamba, can be faster alternatives. Experiments on 432 tasks show that they compare favourably in terms of performance and speed 🎃 arxiv.org/abs/2410.22391

Join Our Research Team in Linz! We are looking for 5 PostDocs and 10 PhDs in Machine Learning working on xLSTM, NLP, robustness, learning theory. Deadline: 04/20/25. More details: jku.at/en/lit-artific… #MachineLearning #DeepLearning #ResearchOpportunities #PhDPositions

Mamba2D: A Natively Multi-Dimensional State-Space Model for Vision Tasks paper: arxiv.org/abs/2412.16146 SSMs are emerging as efficient alternatives to transformers but struggle with spatial dependencies in visual tasks due to biases from their natural language processing origins. Existing methods rely on arbitrary 1D scan directions to process 2D data, which limits effectiveness. Mamba2D introduces a native 2D scan direction that models spatial dependencies more effectively by considering both input dimensions simultaneously. On the ImageNet-1K dataset, Mamba2D achieves performance comparable to prior SSM adaptations for vision tasks, offering a more efficient approach to handling visual inputs.

Transformers can be slow for real-time applications like robotics. We study if modern recurrent architectures, like xLSTM and Mamba, can be faster alternatives. Experiments on 432 tasks show that they compare favourably in terms of performance and speed 🎃 arxiv.org/abs/2410.22391