Infini-AI-Lab

Joint work with @wonderingkrish, @chenzhuoming911, @ChengLuo_lc, @BrianChen112900, @haizhong_zheng, @xxunhuang, Atri Rudra, and @BeidiChen We're grateful for the great work on MonarchAttention as well as concurrent work on VMonarch by @Kling_ai

Result: high-sparsity attention with no visible quality loss—while speeding up attention enough to matter end-to-end for real-time generation. If attention is your bottleneck, MonarchRT is a drop-in path to lower latency. 🧵5/5

Video generation models are improving fast—real-time autoregressive models now deliver high quality at low latency, and they’re quickly being adopted for world models and robotics applications. So what’s the problem? They’re still too slow on consumer hardware. 🚀 What if we told you that we can get true real-time 16 FPS video generation on a single RTX 5090? (1.5-12x over FA 2/3/4 on 5090, H100, B200) Today we release MonarchRT 🦋, an efficient video attention that parameterizes attention maps as (tiled) Monarch matrices and delivers real E2E gains. 📄 Paper: arxiv.org/abs/2602.12271 🌐 Website: infini-ai-lab.github.io/MonarchRT 🔗 GitHub: github.com/Infini-AI-Lab/… 🧵1/n

🚀 New drop on RL scalability & stability!!! Jackpot breaks the bottleneck: not only train with hundreds of stale rollouts — even from a different policy (model). This changes how far we can scale RL. 🔥

For implementation details and limitations of Jackpot, please refer to the paper. We thank the authors for developing Jackpot. @IronSteveZhou, @LiuAtlas89429, @chenzhuoming911, @haizhong_zheng, @BeidiChen [n/n]

Empirically, we evaluate Jackpot 🎰 under diverse and extreme mismatch settings: 🤪 Joint-training: Consistently outperforms TIS across the extreme misalign small→large model RL training setups: Qwen2.5 1.5B→3B, Qwen3 1.7B→4B, Qwen3 1.7B→8B 🤪 Non-two-model-joint-training settings: In highly stale off-policy regimes (large rollout batches), Jackpot enables: • Removing PPO clipping • Convergence rate approaching on-policy training and faster than staleness baseline [5/n]

RL is notoriously unstable under actor–policy mismatch 😥 — a common reality caused by kernel differences, MoE randomness, FP8 rollouts, or asynchronous pipelines. But here’s a crazy thought 🤔 👉 What if you could RL-train a large model using rollouts generated only by a weaker, faster, and completely different model? Sounds doomed from the start? 💩 We are releasing Jackpot 🎰.💡 enabling training Qwen3-8B-Base using only Qwen3-1.7B-Base generated rollouts ✨ Jackpot is surprisingly powerful: • Enables cheap, fast rollouts to train stronger models • Dramatically changes the cost–performance tradeoff of RL training We release Jackpot 🎰 in the following format: 🌔Paper: arxiv.org/abs/2602.06107 🌕Code: github.com/Infini-AI-Lab/… 🌖Blog: infini-ai-lab.github.io/jpt_website/ [1/n]

🎉 M2PO accepted to ICLR! Huge thanks to all collaborators. We’ll release the final version soon.

🚀 InfiniAI Lab @ CMU is hiring Postdocs! We are looking for outstanding postdoctoral researchers in ML systems and security to join InfiniAI Lab at Carnegie Mellon University. Research directions include (but are not limited to): 🤖 AI Agents & RL 🔐 Machine Learning Security 🎥 Video Models 🏗️ AI Systems & Architecture Design We especially encourage candidates interested in applying for the CMU–Bosch Institute (CBI) Postdoctoral Fellowship, which provides strong support for independent, high-impact research: 👉 carnegiebosch.cmu.edu/fellowships/in… 🗓️ CBI application deadline: January 30, 2026 How to apply: Please fill out the form and send us an email via 👉 infini-ai-lab.cmu.edu/vacancies

The most fun part: interpretability 🔍 Token-specific STEM embeddings behave like steering vectors. Even with the same input text, swapping the STEM embedding can meaningfully shift the output distribution 🎛️✨

Where does STEM help most? Big wins on knowledge-heavy benchmarks (ARC-Challenge, OpenBookQA, MMLU)… …but also strong improvements on contextual reasoning (BBH, LongBench) & long-context tasks (NIAH, LongBench). 🧩⏳

Lookup memories are having a moment 😄 The whale 🐋 #deepseek dropped engram… and we dropped up-projections from our FFNs…perfect timing 😅 🥳 Introducing STEM: Scaling Transformers with Embedding Modules 🌱 A scalable way to boost parametric memory with extra perks: ✅ Stable training even at extreme sparsity ✅ Better quality for fewer training FLOPs (knowledge + reasoning + long-context gains) ✅ Efficient inference: ~33% FFN params removed + CPU offload & async prefetch ✅ More interpretable → seamless knowledge editing 🔧🧠 Looking forward to DeepSeek v4… feels like we’ve only scratched the surface of embedding-lookup scaling 👀 📄Paper: arxiv.org/abs/2601.10639 🌐 Website: infini-ai-lab.github.io/STEM 🔗 GitHub: github.com/Infini-AI-Lab/…