Arnaud Autef

Gemini3 Flash was my first release as distillation TL. As it turns out, our bets paid off and Flash is a huge success! Very grateful to work in such a talented team, to @FeinbergVlad for the trust and leadership, and excited to keep pushing: there is so much morecoming ⚡️⚡️⚡️

@goodfellow_ian @daniel_rossett So happy to hear this Ian, welcome back!!

I'd like to thank @daniel_rossett for his help in my recovery from the POTS version of Long COVID. Daniel was key in bringing me back from highly disabled and suffering to being able to do what I want to again. This X account is mostly focused on ML / AI. From that point of view, many of you know that in December 2024, I wasn't able to do the test of time award talk at NeurIPS, even by video call. Daniel started working with me in March 2025. By April, I started to have days of no POTS symptoms, by June I was off all heart rate lowering medications, by September I was back to work. I'm back to full exercise, running, lifting weights, mountain biking, and have even done things I hadn't done before I got sick, like riding Whistler Mountain Bike Park. I'm now getting the word out to help Daniel build a company that will bring this approach to more people.

Excited to share LUCID — a new attention mechanism that improves retrieval and reasoning in long-context LLMs! [1/9]🧵 Here's how it works:

Beyond MuP: 2. Linear Layers and Steepest Descent kexue.fm/archives/11605 The last blog post before the 2026 Spring Festival. Happy Chinese New Year!

@FeinbergVlad Congrats!!! 🐐

This all started with a model that punched way, way, way above its weight class! It's a good thing we're good at doing that :) A special thank you to the NYC + MTV folks that pulled long nights and weekends to make this happen. Working on tech like this is literally my dream!

This wasn't an easy win. We entered the bakeoff later than everyone else. Apple initially said for their evals on public APIs we were behind competitors. The team here is small and also responsible for mainline gemini deliverables concurrently. With some lateral thinking and focussed execution on pre + post, we delivered something far above expectations (I think there was quite a bit of internal surprise too). It took another further QAT invention to get things over the line. Especially with short timelines, it's not so easy to turn compute into a win like this.

Joint Statement: Apple and Google have entered into a multi-year collaboration under which the next generation of Apple Foundation Models will be based on Google's Gemini models and cloud technology. These models will help power future Apple Intelligence features, including a more personalized Siri coming this year. After careful evaluation, Apple determined that Google's Al technology provides the most capable foundation for Apple Foundation Models and is excited about the innovative new experiences it will unlock for Apple users. Apple Intelligence will continue to run on Apple devices and Private Cloud Compute, while maintaining Apple's industry-leading privacy standards.

I can confidently say I never want to work with anyone who publicly shits on their former team without any gratitude like this.

I’m launching Tutorial II for Physics of Language Models. Many people focus on large-scale results. This tutorial is about why those results are often artifacts of noise — and how to eliminate that noise at the design level. The first video (Part 4.1a, 1 hour) is the most important one. It focuses on methodology, not benchmarks: – how real-life pretraining can be “cheated”, – why academic-scale experiments are noisy, – and most importantly, how to design a versatile, skill-pure synthetic pretraining playground. I explain why our five synthetic tasks are designed the way they are, and how GPT2-small-scale (100M) models can reveal architectural truths that 8B models trained on 1T tokens often fail to expose reliably. This methodology is the backbone of the entire Physics series. ▶️ First video: Part 4.1a — methodology & playground design 🔜 Second video: Part 4.1b — architectural principles from the playground 🔜 Third video: Part 4.2 — when the playground reshapes real-life pretraining (You can find it via my profile.)

last week, LMSYS quietly dropped one of the cleanest “learn LLM inference” projects out there > Mini-SGLang it’s basically a production-grade serving stack compressed into ~5k lines of readable Python in this project, you’ll implement the core mechanics behind modern LLM inference systems: > FlashAttention-3 + FlashInfer > fully working kernels > tensor parallelism > scale decode cleanly across GPUs > chunked prefill > serve long context without blowing memory > JIT-compiled CUDA ops > see how kernels are stitched at runtime > overlap scheduling > hide CPU orchestration behind GPU compute > radix cache > reuse KV cache across shared prefixes > OpenAI-compatible API > serve models like production systems do what makes it special > online + offline inference > streaming output > overlap scheduling actually implemented > no toy shortcuts everything you learn here transfers directly to real serving stacks you’ll understand > how schedulers work under concurrency > where latency actually hides > how KV reuse changes throughput > why chunking beats naive prefill > how TP interacts with cache + decode the design philosophy > small codebase > fully type-annotated > modular > debuggable > meant to be read end-to-end run it yourself > single-GPU: small models > multi-GPU: 70B-class models > same codepath, same API if you want to actually learn LLM inference > not slides > not diagrams > read the code (with your favorite LLM) > step through the scheduler > trace the cache > break it > fix it this is what modern LLM serving looks like

Comment as-tu oublié ça si vite?

1/ @johnschulman2 mentioned that the main important purpose of value functions/models in RL is variance reduction—but in current LLM-RL tasks, they aren't delivering much. What if we could get token-level variance reduction without training a value model at all? 🧵👇richardli.xyz/optimal-token-…

I exactly know how to make Arena allocator work exciting for people to read 2014: This was my first project that I colead with Chris Fallin with many support from Sanjay and Jeff (while never meeting in person) as well as Greg Pike. This project caused raining of spot bonuses at google at me, and ability to move to any team which i spent on moving to research and working on optimization (not code! But neural network training) There were a tonne of clever optimizations at Google scale. A trip down memory lane, and all of this is OSS - I can proly look up and document it. I was increasingly just spending my time optimizing google servers (largely from boredom given I wanted to work on ML and didnt have the cred to do ML yet). google wide profiling shows anybody at the company where the cpu time is spent and code is all open for anyone to read. This lead to finding that most of Google servers (sometimes upto 50%) spent time on allocated, deallocate, serialize and deserialize. Tcmalloc did a good job, but protobuf design with its many level trees meant death by a hundred thousand cuts. Arena allocator sounded like a good idea, so I picked up some prototype that was not working yet from Liu Jisi and optimized it. First on the menu was speeding up thre destructor chain with destructor skippable trait/ SIFNAE - this showed signs of life and we sent the benchmarks to the protobuf mailing list which got everyone interested. In a short 4 month sprint during the summer of 2024, all I did was write code to get Arena work for Google with Chris. Good times! It has stayed the test of time at Google. Every single rpc goes through it, probably saved google O(10b) dollars on machine costs. Here is the very fast arena allocator github.com/protocolbuffer… Code still looks nice! Few clever things - PerThread Arena for avoiding contention - AlignedAlloc Fast Path! - Skipping chain destructors for large depth - custom string instead of std::string which we got a lot of flak for, but was needed for performance We did several clever things to integrate it to existing protobuf and make sure no servers fail over * low level optimization of generated code to balance binary size and run time. This is where Geoff Pike showed his brilliance! * Sanjay’s continued bithacking brilliance with unknownfieldset packing of arena ptr to save 4 bytes per protobuf which was a blocker in integration Martin who was initially using it for Play Servers fixed one of the perf problems inside arena but I don’t remember it yet.

Yann is just plain incorrect here, he’s confusing general intelligence with universal intelligence. Brains are the most exquis​ite and complex phenomena we know of in the universe (so far), and they are in fact extremely general. Obviously one can’t circumvent the no free lunch theorem so in a practical and finite system there always has to be some degree of specialisation around the ​target distribution that is being learnt. But the point about generality is that in theory, in the Turing Machine sense​, the architecture of ​s​uch a general system is capable of learning anything computable given enough time and memory​ (and data), and the human brain (and AI foundation models) are approximate Turing Machines. Finally, with ​regards to ​Yann's comments about chess players, it’s amazing that humans could have invented chess ​in the first place (and all the other ​a​spects ​o​f modern civilization ​from science to 747s!) let alone get as brilliant at it as someone like Magnus. He may not be ​strictly optimal (after all he has finite memory and limited time to make a decision) but it’s incredible what he and we can do with our brains given they were evolved for hunter gathering.

This is what we've been coking for the last 9 months: make MoEs training goes ~2x faster and ~2x less memory! Highlights: - MoE typically takes the most time and memory in modern models. Turns out one can mathematically rewrite the MoE backward pass to reduce the activation mem you need to store in the fwd by ~2x, resulting in the same gradients with no extra matmul recomputation. I really like this result, as it combines both algorithmic and systems insights. - Analyzing bottlenecks in MoE layer leads to a natural optimization stragegy: reduce mem reads/writes as much as possible! Gathering the input for fwd and output grad for bwd can sometimes take as much time as the grouped GEMMs. We fuse gather with grouped GEMM + overlap mem access and compute to make the whole layer goes ~2x faster. - Computing top-k for expert routing can take surprisingly long, ~15-20% of the whole MoE layer! Standard top-k impl uses radix top-k algo, great for large k but suboptimal for small k. We rewrote top-k using bitonic top-k algo, and it's sometimes 20-30x faster than pytorch's top-k! All the main kernels are written in Cute-DSL so they should be easy to extend (and install :D). Hopper kernels are out, Blackwell kernels are just about ready. MoE models used to be 2x less hardware-efficient to train, hopefully Sonic-MOE will change that.

⚡️⚡️⚡️

⚡️⚡️⚡️ also if you dont believe lmarena, please try it for yourselves and let us know how it feels 🫡 ai.dev

⚡️⚡️⚡️

Flash 3.0 is out! Culmination of years of high-conviction infra & science investment (we knew it'd be this good a priori). I truly believe no other place trains these models the way we do. I'm lucky to get to work so intimately with the team that made this happen. We aren't slowing down.