Julia Kempe

@chi_t_williams @ziv_ravid Way before

@ziv_ravid Quick Q: was this recorded before or after the unit distance result?

Math is starting to fall — so what's next? 🎙️ New episode of The Information Bottleneck is out! We've all seen the recent wave of Erdős problems being solved by frontier models, and the question now is what it actually means for the future of mathematics, and for AI research more broadly. We sit down with @KempeLab - Professor at NYU's Center for Data Science and researcher at Meta FAIR's Foundations of Reasoning team, to dig into exactly that. Julia makes the case that math is the next Go. With formal verification and LLM agents that can propose, formalize, and check proofs at scale, a new industry of automated mathematical discovery is closer than most mathematicians believe. We also get into: → Why physics is harder than math → Model collapse, synthetic data, and what's left to squeeze from the internet → Scaling limits, energy costs, and where academia still has the edge → How to advise PhD students when Claude can already do their first-year work → AI safety, agent security, and the Wild West of deployed agents → Why the Renaissance researcher is finally back One of our favorite conversations yet. Listen now 👇

The 2nd Sci4DL (@scifordl) workshop at #ICLR2026 in Rio drew a packed room. Organized by CDS PhD alumni @ZKadkhodaie & @LotfiSanae, CDS Instructor @FlorentinGuth, and CDS-associated Prof. Eero Simoncelli, with CDS Silver Prof. @KempeLab as a speaker. scienceofdlworkshop.github.io/2026/

@scottnarmstrong ... and of all further derived statements end lemmata!! Intergenerationally...

Even this is far too lenient. If you really cared about scientific integrity you’d not only do a retroactive banning of their publications, but you’d axiomatize the negation of all the theorems they proved.

CDS Silver Prof @KempeLab working with @arnal_charles, @TacoCohen, Vivien Cabannes, and Remi Munos studied how LLMs can train more efficiently by reusing past experience instead of constantly generating new data. Accepted to the ICML '26 conference. arxiv.org/abs/2604.08706

3/3 Scaling of foundation models & large-scale engineering continues, but further progress in machine intelligence will require new ideas & breakthroughs. I believe academia will continue to play a key role, particularly through published and opensource research. Exciting times!

2/3 I am deeply grateful for the opportunity to collaborate with so many amazing colleagues at FAIR and MSL @AIatMeta. I also want to thank FAIR leadership, past and present, especially @ylecun, @jpineau1, @NailaMurray, David Lopez Paz, @rob_fergus for letting us explore.

1/3 My time at Meta FAIR will soon come to a close. I joined nearly two years ago full-time to help advance LLM reasoning. It has been a remarkable journey working with and leading an exceptionally talented team.

@arnal_charles Lucky for our team to have you, Charles!

Very excited to be joining Meta Superintelligence Labs as a Research Scientist! I’ll be continuing my work on RL and AI for maths with @KempeLab, Rémi Munos, and my longtime partner in crime, Vivien Cabannes.

@scifordl @ZKadkhodaie An amazing, super well organised (and popular!) workshop! Thanks to all the organisers!

Closing remarks: @ZKadkhodaie is happy!

"Insights into LLMs reasoning" by Julia Kempe @KempeLab

Tomorrow's Frontiers in Assciative Memory workshop at ICLR26 (room 201C) will be an exciting event!

Our work today (Sat) @iclr2026: ORAL: - OpenApps 3:39pm 204A/B POSTERS: -10:30am: Soft Tokens, Hard Truths, P3-#1020 -10:30am: OpenApps, P3-#308 - 3:15pm: How reinforcement learning after next-token prediction facilitates learning, P3-#806 Come to discuss!

Heading to Rio for #ICLR2026 (there 24-27): Giving a talk & on panel #Sci4DL workshop Today's (Wed) poster on interpretability: • From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers Thu Apr 23, 10:30 AM-1:00 PM | P4-#4002

Excited to be at #ICLR2026 in Rio this week! Presenting “Soft Tokens, Hard Truths” Saturday 10.30am (Pavilion 3, P3-#1020). Feel free to DM me to chat about self-improvement, reasoning, code gen. I’m also on the job market for industry research positions.

love this replay buffer paper from Meta: arxiv.org/abs/2604.08706 "methods like PPO or GRPO typically operate as on-policy as possible, meaning rollouts are generated, used for a single gradient update, and immediately discarded." this is crazy and we shouldn't do this!

Efficient RL Training for LLMs with Experience Replay "Empirically, we show that a well-designed replay buffer can drastically reduce inference compute without degrading – and in some cases even improving – final model performance, while preserving policy entropy."

(1/9) Experience replay can cut LLM RL training compute by up to ~40% (without hurting final accuracy—and sometimes improving it). Paper: arxiv.org/abs/2604.08706

Experience replay of LLM RL. Hyperparameters for replay buffers, buffer sizes N, number of new rollouts R, minibatch size B depend on compute imbalance of rollout generation and training, and task-specific statistics. It could be hard to optimize this well.

@KempeLab and I just vibe-coded a Lean 4 formalization of elliptic De Giorgi–Nash–Moser theory. This is a cornerstone of modern elliptic PDE: local boundedness, weak Harnack, Harnack, and Hölder regularity for weak solutions with merely bounded measurable coefficients.