Alessandro Betti

#LOT Spring school @biggiobattista presenting the lecture "Machine Learning Security: Lessons Learned and Future Challenges" @Cossu94 @v_lomonaco @StefanoMelacci @aleitteb

🚀 Learning Over Time (LOT) Spring School 📅 24-27 March 2025 | 📍 Siena, Italy 🔗sites.google.com/unisi.it/lot-s… Organizers: @StefanoMelacci ,@v_lomonaco @Cossu94 ,@aleitteb @TiezziMatteo #ContinualLearning #CL #LifelongLearning #CollectionlessAI

🚀 We’ve launched a new site for Collectionless AI, sharing our take on next-gen AI/ML! 🚀 Explore our papers, events, and more! cai.diism.unisi.it #CollectionlessAI #ContinualLearning #LifelongLearning #MachineLearning #AI 🌐 @StefanoMelacci @KleineBottleM @TiezziMatteo

#CoLLas2024 folks 🐨 Temperature rising for the coming sessions 🔥🐨

Hurry up for @CoLLAs_Conf registration!

📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" arxiv.org/abs/2406.09062 a joint work by @TiezziMatteo , M. Casoni, @aleitteb , M.Gori (@KleineBottleM) and @StefanoMelacci

We also summarized the most common #benchmarks and settings for long sequence processing and #longrange dependencies arxiv.org/abs/2406.09062

3⃣ RNNs evolutions, such as xLSTMs (@gklambauer), ODE-inspired RNNs (Neural Oscillators - @tk_rusch) as well as novel paths for 4⃣ Learning in RNNs, such as modern evolutions of RTRL (@KhurramJaved_96) or bio-inspired methods (@BerenMillidge) arxiv.org/abs/2406.09062

2️⃣ Deep State-space Models #SSMs such as #S4, #S5, #Mamba, #Griffin etc. a line of work by @_albertgu @tri_dao @SamuelMLSmith, @orvieto_antonio and many others arxiv.org/abs/2406.09062

We described several trends: 1️⃣ #Transformers embraced Recurrence exploiting a stateful recurrent representation: *️⃣ all the latest models RWKV4/5/6 by @BlinkDL_AI *️⃣ #RetNet by @donglixp et al. are competitive with quadratic self-attention! arxiv.org/abs/2406.09062

Goal: efficiently 𝐩𝐫𝐨𝐜𝐞𝐬𝐬 𝐥𝐨𝐧𝐠 𝐬𝐞𝐪𝐮𝐞𝐧𝐜𝐞𝐬 + 𝐩𝐫𝐞𝐬𝐞𝐫𝐯𝐞 𝐥𝐨𝐧𝐠 𝐭𝐞𝐫𝐦 𝐝𝐞𝐩𝐞𝐧𝐝𝐞𝐧𝐜𝐢𝐞𝐬 TL;DR #Transformers + #RNNs, #DeepStatespace models ➡️ progressive confluence of previously disjoint architectures arxiv.org/abs/2406.09062

🧐Interested in the details of the latest SOTA architectures (#RWKV , #RetNet, #Mamba, #Griffin) for long sequences? 📢 "State-space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era" arxiv.org/abs/2406.09062

📢Jay McClelland, co-author of the seminal 'Parallel Distributed Processing' book 📚, is set to be our keynote speaker at #CoLLas2024! 🎉

‼️Short survey paper‼️ 📢"On the Resurgence of Recurrent Models for Long Sequences: Survey and Research Opportunities in the Transformer Era" by @TiezziMatteo M.Casoni @aleitteb @_masaccio_ @KleineBottleM @StefanoMelacci #Transformers #SSMs arxiv.org/abs/2402.08132

🚨1 month left until the #CoLLAs2024 paper submission deadline!🚨 Don’t miss your opportunity to contribute to the field of lifelong learning. Finalize your research and submit by 15 Feb 2024! 📄 Prepare your submission: lifelong-ml.cc/Conferences/20… 🗓️ Abstract deadline: 09 Feb 2024

It is sad to lose the DeepMind office in Edmonton to the Tech layoffs and looming recession. But AI is not going away, and I am more focused than ever on the Alberta Plan for AI research. arxiv.org/abs/2208.11173

📣 Last week I presented "Foveated Neural Computation" @ECMLPKDD ➡️ Paper: 2022.ecmlpkdd.org/wp-content/upl… ➡️ Code: github.com/sailab-code/fo… A thread👇🧵 In some visual tasks, such as image classification, salient information is distributed in regions of limited size 🧐 1/n

(8/n) @TiezziMatteo, @MarulloSimone, Lapo Faggi, @enricomeloni_ai, @aleitteb and @StefanoMelacci investigated the effectiveness of gravitational models in an open-set class-incremental setting, by imposing stochastic coherence along the visual scanpath. arxiv.org/abs/2204.12193

Mi raccomando, durante gli incontri parentali delle feste, oltre a stare attenti, non appena lo zio dice “siero speri..” siate saggi 😁

My Secret Vault is a CHEF'S DREAM youtu.be/_ff9EwPIJE8 via @YouTube