Kamesh Krishnamurthy

immensely proud to share 2 of my bros' work, Hybrid Associative Memories by @kamesh_ai and @leonlufkin It's basically SSM and Attention merged into one layer, where the attention is also sparse like DSA. Except you pretrain with the sparsity. It's lowkirk based (1/N) 👇

@ZyphraAI releases research on a new way to build hybrid models. We introduce a new architecture leveraging the complementary strengths of Transformers and RNNs for greater flexibility and performance than existing approaches. We call it Hybrid Associative Memory (HAM). 🧵

Hybrid layers that combine RNNs and Transformers in a complementary way. @Andr3yGR @CPehlevan @SuryaGanguli @Gautam_Reddy_N @EkdeepL @Nick__Alonso @SonglinYang4

Project authors: @leonlufkin @TomFigliolia @BerenMillidge @kamesh_ai Technical paper: arxiv.org/abs/2603.22325 Technical blog: zyphra.com/post/ham