Reece Shuttleworth

@StefanoErmon Its been great working with @StefanoErmon and the @_inception_ai team on Mercury 2! I'm excited to keep pushing the frontier 🚀

Mercury 2 is live 🚀🚀 The world’s first reasoning diffusion LLM, delivering 5x faster performance than leading speed-optimized LLMs. Watching the team turn years of research into a real product never gets old, and I’m incredibly proud of what we’ve built. We’re just getting started on what diffusion can do for language.

We just launched Mercury 2 at @_inception_ai! 5x the model speed, 5x the brainrot speed.

@beffjezos It already ate images and videos. Language just took a little longer to chew.

The more structure a language has, the faster diffusion can run. Code fits that profile. Code has plenty of it. Listen to @justkharbanda on how diffusion unlocks speed for real-world coding workloads. #Diffusion #AIInfrastructure #DeveloperTools

Diffusion clicked for me when I read about score-based models, a line of work pioneered by @StefanoErmon (et al.) at Stanford. So it was a full-circle moment to collab with him and @_inception_ai on a video about training & sampling techniques for making diffusion LLMs faster.

"It's now or never." Our CEO @stefanoermon on why he started Inception – featured this week in @WSJ. We're building dLLMs that generate tokens in parallel. Faster. More efficient. More controllable. This is the moment. Thanks for the coverage @KateClarkTweets wsj.com/tech/ai/these-…

Excited to have this work included in the PEFT library! PR: github.com/huggingface/pe…

Diffusion will obviously work on any bitstream. With text, since humans read from first word to last, there is just the question of whether the delay to first sentence for diffusion is worth it. That said, the vast majority of AI workload will be video understanding and generation, so good chance diffusion is the biggest winner overall. Also means that the ratio of compute to memory bandwidth will increase.

Huge thank you to Pratyusha Sharma (@pratyusha_PS), Jacob Andreas (@jacobandreas), and Antonio Torralba for their collaboration on this work! See code here: github.com/reeceshuttle/i…

Really cool to see @thinkymachines exploring similar ideas around LoRA recently! Check out our paper to see our other detailed investigations of diverse topics: How do LoRA initialization and learning rate impact learning? What role does LoRA’s alpha parameter and the product-of-matrices parameterization play in training dynamics observed? Plus mathematical explanations of this phenomenon and more!

🧵 LoRA vs full fine-tuning: same performance ≠ same solution. Our NeurIPS ‘25 paper 🎉shows that LoRA and full fine-tuning, even when equally well fit, learn structurally different solutions and that LoRA forgets less and can be made even better (lesser forgetting) by a simple intervention! Read on for behavioral differences (forgetting, continual learning) and other analysis! Paper: arxiv.org/pdf/2410.21228 (1/7)

1/7 Wondered what happens when you permute the layers of a language model? In our recent paper with @tegmark, we swap and delete entire layers to understand how models perform inference - in doing so we see signs of four universal stages of inference!