Matteo Tiezzi

🧐Interested in the details of the latest SOTA architectures (#RWKV , #RetNet, #Mamba, #Griffin) for long sequences? 📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" arxiv.org/abs/2406.09062

From unknown whispers in the Prologue to the first footsteps into an unknown world in Chapter One — the journey begins. Shadows stir, secrets breathe. Are you ready to turn the first page?" #amwriting #fantasybooks #fantasy #DarkFantasy #mystery #horrortale #shortstory Links 👇

📣"State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" is available open-access in Elsevier Neural Networks! Updated version with insights on #Mamba2, #RWKV 5/6/7, #DeltaNet, #GLA etc. 📜Check it out: sciencedirect.com/science/articl…

A great thread that showcases the main intuitions behind our #FovEx paper Paper: 🔗 link.springer.com/article/10.100… Technical Report: 🔗arxiv.org/abs/2408.02123 Github: 📷 github.com/mahadev1995/Fo…

🚨 Our #survey "Back to recurrent processing at the crossroad of #transformers and state-space models" is out in Nature Machine Intelligence! 🔗 nature.com/articles/s4225… Read it here: rdcu.be/el85K @StefanoMelacci @KleineBottleM (M. Gori)

@julien_siems @leloykun @jyo_pari In our DeltaProduct work we also add a bit of theory to DeltaNet, showing that it can solve Dihedral groups, which are the groups of symmetries of regular polygons, with only two layers. This includes S3 (symmetries of the equilateral triangle).

1/9 There is a fundamental tradeoff between parallelizability and expressivity of Large Language Models. We propose a new linear RNN architecture, DeltaProduct, that can effectively navigate this tradeoff. Here's how!

#LOT startup day with @Continual_IST (@v_lomonaco), Motus ML (@GiacomoZiffer ) and @FocoosAI (@fcdl94) @Cossu94 @StefanoMelacci @aleitteb

#LOT very interesting interactive lecture by @claudiogallicc1 "Reservoir Computing Methods for Time Series Classification" @Cossu94 @v_lomonaco @StefanoMelacci

#LOT Spring School @eliricci_ on "Beyond Zero-Shot Generalization: Strengthening Vision-Language Models with Adaptation & Personalization" cai.diism.unisi.it/school/ @StefanoMelacci @v_lomonaco @Cossu94

#LOT Spring school @biggiobattista presenting the lecture "Machine Learning Security: Lessons Learned and Future Challenges" @Cossu94 @v_lomonaco @StefanoMelacci @aleitteb

#LOT Spring school Extremely honoured to introduce the second talk of the day, by the great @orvieto_antonio, on "New Avenues in Long-Sequence Processing without Attention" cai.diism.unisi.it/school/ @StefanoMelacci @Cossu94 @v_lomonaco @aleitteb

#LOT spring school we continue our incredible lineup of speakers with Alexei Efros and the talk titled "Test-Time Training: Toward a Post-Dataset World", @StefanoMelacci @Cossu94 @v_lomonaco @aleitteb cai.diism.unisi.it/school/

Art, nature and Artificial Intelligence at LoT Spring school which just started. With 50+ students, we are fully booked and eager to learn (over time). @StefanoMelacci @v_lomonaco @TiezziMatteo Alessandro Betti, Marco Gori

The first edition of Learning Over Time (LOT) Spring School is starting! @StefanoMelacci kicking off the school at the wonderful Certosa di Pontignano, Italy Organized by @StefanoMelacci, @v_lomonaco , @Cossu94 , @aleitteb , and myself cai.diism.unisi.it/school/

Working in #Multimedia, #ComputerVision, #AI? The Deadline for #ACMMM2025 is approaching fast! Don't forget to follow the new official profile for updates! x.com/acmmm_social

If you want to achieve AGI, you need to solve continual learning! @CoLLAs_Conf is the premier venue for continual learning research! Check this blog post to read more about CoLLAs 2025!

🚀 Announcing ASAP: asap-seminar.github.io! A fully virtual seminar bridging theory, algorithms, and systems to tackle fundamental challenges in Transformers. Co-organized by @simran_s_arora @Xinyu2ML @HanGuo97 Our first speaker: @heyyalexwang on Test-time Regression

Attention has been the key component for most advances in LLMs, but it can’t scale to long context. Does this mean we need to find an alternative? Presenting Titans: a new architecture with attention and a meta in-context memory that learns how to memorize at test time. Titans are more effective than Transformers and modern linear RNNs, and can effectively scale to larger than 2M context window, with better performance than ultra-large models (e.g., GPT4, Llama3-80B).

🚀 Learning Over Time (LOT) Spring School 📅 24-27 March 2025 | 📍 Siena, Italy 🔗sites.google.com/unisi.it/lot-s… Organizers: @StefanoMelacci ,@v_lomonaco @Cossu94 ,@aleitteb @TiezziMatteo #ContinualLearning #CL #LifelongLearning #CollectionlessAI

😸 Join the LOT Spring School with an incredible speaker lineup: @biggiobattista , @NicoloCB, @eliricci_, @orvieto_antonio , @KleineBottleM , Tinne Tuytelaars 🔗sites.google.com/unisi.it/lot-s… 🔗 Pre-registrations are now open, secure your spot today: sites.google.com/unisi.it/lot-s…

🚀 Learning Over Time (LOT) Spring School 📅 24-27 March 2025 | 📍 Siena, Italy 💡 Are you considering models that continuously adapt over time instead of learning "offline" from pre-designed-huge collections of data? 🚀 🔗sites.google.com/unisi.it/lot-s…