Julia Kempe
241 posts
Julia Kempe
@KempeLab
Silver Professor @ NYU Courant & CDS AmiLabs Research in Machine Learning & AI, ex Director @ MetaFAIR, past in Quantum Comp. & Finance Posts my own.
Katılım Nisan 2024
266 Takip Edilen3.5K Takipçiler
ICML Posters from our group - if interested, come and dicuss!
🎉Spotlight: Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
Wed July 8 5:00-6:45pm Hall A Location: #3713
Efficient RL Training for LLMs with Experience Replay
Thu July 9 10:30am-12:15pm Hall A #2201
Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation
Thu July 9 2:30-4:15pm Hall A #1909
What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
English
Julia Kempe retweetledi
I'll be at #ICML2026 to present our *spotlight* work! TLDR: LLMs can learn to self-generate curricula for problems they can't yet solve, using self-play with meta-RL.
Please reach out to chat about self-improving agents, synthetic data & environments, curriculum learning, or anything else!
We've updated the paper with some fun additions ⬇️
Shobhita Sundaram @ ICML@shobsund
Can a model learn to break its own reasoning plateau? In our new paper, we show that LLMs can be taught with meta-RL to generate their own "stepping stones" that kickstart learning on hard math problems (0/128 success rate) where direct RL fails. Paper 📝: arxiv.org/abs/2601.18778 Blog post 🌐: ssundaram21.github.io/soar/ (1/n)
English
Our Physics of Learning Simons Collaboration @ ICML!
Surya Ganguli@SuryaGanguli
Check out all the amazing work from our @SimonsFdn Collaboration on the Physics of Learning and Neural Computation (physicsoflearning.org) presented at the main meeting of @ICMLconf #ICML2026 Tuesday Efficient Learning of Compositional Targets with Hierarchical Spectral Methods,Hugo Tabanelli, Yatin Dandi, Luca Pesce, and Florent Krzakala icml.cc/virtual/2026/p… CompleteP for RL: Maintaining Feature Learning When Scaling Deep Reinforcement Learning M Ganesh Kumar, Adam Lee, Blake Bordelon , Cengiz Pehlevan icml.cc/virtual/2026/p… Universal One-third Time Scaling in Learning Peaked Distributions Yizhou Liu, Ziming Liu, Cengiz Pehlevan, Jeff Gore icml.cc/virtual/2026/p… Wednesday A Noise Sensitivity Exponent Controls Large Statistical-to-Computational Gaps in Single- and Multi-Index Models, Leonardo Defilippis, Florent Krzakala, Bruno Loureiro, Antoine Maillard icml.cc/virtual/2026/p… Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws Fabrizio Boncoraglio, Vittorio Erba, Emanuele Troiani, Yizhou Xu, Florent Krzakala, Lenka Zdeborová icml.cc/virtual/2026/p… A Solvable High-Dimensional Model Where Nonlinear Autoencoders Learn Structure Invisible to PCA While Test Loss Misaligns With Generalization Vicente Mendes, Lorenzo Bardone, Cédric Koller, Jorge Medina Moreira, Vittorio Erba ⋅ Emanuele Troiani, Lenka Zdeborova icml.cc/virtual/2026/p… Deep networks learn to parse uniform-depth context-free languages from local statistics Jack T. Parley, Francesco Cagnetta, Matthieu Wyart icml.cc/virtual/2026/p… Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling Indranil Halder, Cengiz Pehlevan icml.cc/virtual/2026/p… On the Existence of Consistent Adversarial Attacks in High-Dimensional Linear Classification Matteo Vilucchio, Lenka Zdeborova, Bruno Loureiro icml.cc/virtual/2026/p… Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation Zier Mensch, Lars Holdijk, Samuel Duffield, Maxwell Aifer, Patrick Coles, Max Welling, Miranda C. N. Cheng icml.cc/virtual/2026/p… Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability Shobhita Sundaram, John Quan, Ariel Kwiatkowski, Kartik Ahuja, Yann Ollivier, Julia Kempe icml.cc/virtual/2026/p… Thursday Deriving Neural Scaling Laws from the Statistics of Natural Language Francesco Cagnetta ⋅ Allan Raventos ⋅ Surya Ganguli ⋅ Matthieu Wyart icml.cc/virtual/2026/p… Symmetry in language statistics shapes the geometry of model representations Dhruva Karkada, Daniel Korchinski, Andres Nava, Matthieu Wyart, Yasaman Bahri icml.cc/virtual/2026/p… A Random Matrix Perspective on the Consistency of Diffusion Models Binxu Wang, Jacob A Zavatone-Veth, Cengiz Pehlevan icml.cc/virtual/2026/o… Hyperparameter Transfer with Mixture-of-Expert Layers Tianze Jiang, Blake Bordelon, Cengiz Pehlevan, Boris Hanin icml.cc/virtual/2026/p… Analytic Bijections for Smooth and Interpretable Normalizing Flows Mathis Gerdes, Miranda C. N. Cheng icml.cc/virtual/2026/p… Efficient RL Training for LLMs with Experience Replay Charles Arnal, Vivien Cabannnes, Taco Cohen, Julia Kempe, Remi Munos icml.cc/virtual/2026/p… Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text Generation Dhrupad Bhardwaj, Julia Kempe, Tim G. J. Rudner icml.cc/virtual/2026/p… What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT Yunzhen Feng, Julia Kempe, Cheng Zhang, Parag Jain, Anthony Hartshorn icml.cc/virtual/2026/p… From Kepler to Newton: Inductive Biases Guide Learned World Models in Transformers Ziming Liu, Surya Ganguli, Andreas Tolias icml.cc/virtual/2026/p…
English
Check out our new paper on internalization: the process of gradually "absorbing" chain of thought computations during training. Our results show that internalization can work for problems that are computationally hard to learn directly. We carefully study method and task specific factors that determine internalization success. To learn more, see arxiv.org/abs/2606.20937.
With @nikostsilivis @nirmitj_ @_rkomma Nati Srebro.
@NYUDataScience @TTIC_Connect
English
Julia Kempe retweetledi
📢 Just announced!
Join us for the #KempnerInstitute workshop “Learning Dynamics in Natural and Artificial Intelligence: Evolution, Adaptation, and the Foundations of Efficient Learning.”
Learn more, register, or submit an abstract 👉 bit.ly/3QwJlHR
English
Submit your work to the Methods and Opportunities at Small Scale workshop at COLM. Deadline 6/30. This promises to be a super interesting workshop!
MOSS@MOSS_workshop
📢 CfP for the 2nd version of MOSS at @COLM_conf! sites.google.com/view/moss-colm… (Deadline: 6/30) We welcome submissions on small-scale research for algorithmic innovation and scientific understanding across training, architecture, data, evaluation, interpretability, safety, and more!
English
I am excited to share that I am joining @amilabs as Director of Research, Paris, working with @ylecun and an exceptional founding team.
Further progress in machine intelligence will require not only scaling foundation models and large-scale engineering, but also new ideas and breakthroughs.
This is what makes AmiLabs such a unique and exciting place to build. Its focus on world modeling — systems that can learn richer representations of the real world, reason, plan, and learn from interaction — is a long-term research direction I am deeply excited about.
I am particularly looking forward to working with extraordinary co-founders @sainingxie, @lxbrun, @michaelrabbat, @pascalefung, @mavenlin, @laurentsolly, and the amazingly talented AmiLabs team, to helping build the research organization and to the journey ahead.
English
We did it!
Thrilled to announce that with my team at FAIR Meta we released 25+ auto-formalized mathematics textbooks covering analysis, algebra, geometry, topology, combinatorics, probability, statistics, PDEs, number theory, and theoretical computer science - the largest such effort to date.
Charles Arnal@arnal_charles
Our team at @AIatMeta is excited to announce ATLAS: one of the largest automated formalization efforts to date. ATLAS contains Lean 4 formalizations of both statements and proofs from 25+ mathematics textbooks, spanning dozens of domains, for a total of 500k lines of code. We are also releasing a flexible formalization harness and a companion paper. External contributions are welcome! Joint work spearheaded by our amazing PhD student Ahmad Rammal (@Ahmad3Rammal), together with Niket Patel (@niketnpatel ), Fabian Gloeckle (@FabianGloeckle), Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi), Julia Kempe (@KempeLab), Vivien Cabannes, and myself from @AIatMeta, @NYUDataScience , and Ecole des Ponts. This is an ongoing effort; more details in the thread below. (1/9)
English
@ziv_ravid Quick Q: was this recorded before or after the unit distance result?
English
Julia Kempe retweetledi
Math is starting to fall — so what's next? 🎙️
New episode of The Information Bottleneck is out!
We've all seen the recent wave of Erdős problems being solved by frontier models, and the question now is what it actually means for the future of mathematics, and for AI research more broadly.
We sit down with @KempeLab - Professor at NYU's Center for Data Science and researcher at Meta FAIR's Foundations of Reasoning team, to dig into exactly that.
Julia makes the case that math is the next Go. With formal verification and LLM agents that can propose, formalize, and check proofs at scale, a new industry of automated mathematical discovery is closer than most mathematicians believe.
We also get into:
→ Why physics is harder than math
→ Model collapse, synthetic data, and what's left to squeeze from the internet
→ Scaling limits, energy costs, and where academia still has the edge
→ How to advise PhD students when Claude can already do their first-year work
→ AI safety, agent security, and the Wild West of deployed agents
→ Why the Renaissance researcher is finally back
One of our favorite conversations yet.
Listen now 👇
English
Julia Kempe retweetledi
The 2nd Sci4DL (@scifordl) workshop at #ICLR2026 in Rio drew a packed room.
Organized by CDS PhD alumni @ZKadkhodaie & @LotfiSanae, CDS Instructor @FlorentinGuth, and CDS-associated Prof. Eero Simoncelli, with CDS Silver Prof. @KempeLab as a speaker.
scienceofdlworkshop.github.io/2026/
English
@scottnarmstrong ... and of all further derived statements end lemmata!! Intergenerationally...
English
Even this is far too lenient. If you really cared about scientific integrity you’d not only do a retroactive banning of their publications, but you’d axiomatize the negation of all the theorems they proved.
TotientQuotient@t0tientqu0tient
Funny how people complain that the new arXiv policy is too harsh. If anything, it's still way too lenient. Getting caught with AI slop should result in a permanent, publicly viewable (in the form of a hall of shame) ban and retroactive redaction of all previous arXiv submissions.
English
Julia Kempe retweetledi
CDS Silver Prof @KempeLab working with @arnal_charles, @TacoCohen, Vivien Cabannes, and Remi Munos studied how LLMs can train more efficiently by reusing past experience instead of constantly generating new data.
Accepted to the ICML '26 conference.
arxiv.org/abs/2604.08706
English
3/3 Scaling of foundation models & large-scale engineering continues, but further progress in machine intelligence will require new ideas & breakthroughs. I believe academia will continue to play a key role, particularly through published and opensource research. Exciting times!
English
2/3 I am deeply grateful for the opportunity to collaborate with so many amazing colleagues at FAIR and MSL @AIatMeta.
I also want to thank FAIR leadership, past and present, especially @ylecun, @jpineau1, @NailaMurray, David Lopez Paz, @rob_fergus for letting us explore.
English
1/3 My time at Meta FAIR will soon come to a close. I joined nearly two years ago full-time to help advance LLM reasoning.
It has been a remarkable journey working with and leading an exceptionally talented team.
English
Very excited to be joining Meta Superintelligence Labs as a Research Scientist!
I’ll be continuing my work on RL and AI for maths with @KempeLab, Rémi Munos, and my longtime partner in crime, Vivien Cabannes.
English