Jonathan @SF

Post-training LLMs is like mixing a cocktail: Too much easy data → no learning Too much hard data → instability Wrong balance → collapse And today, we mix it by hand. What if the data mixture could be learned instead of hand-tuned? arxiv.org/abs/2602.20532 🧵👇

@MimeeXu Thanks Mimee!!

@lightetal Congrats Jonathan!

Excited to share that our paper was selected for an oral at the #ICLR2026 LLM Reasoning Workshop!

@yisongyue @YuanSui123 @GeelingC @evermind @Caltech @YueLabCaltech Congrats @YuanSui123 @GeelingC !!!

Congratulations to @YuanSui123 & @GeelingC for rocking out at the @evermind Genesis competition! 🚀 We are building some amazing AI Agents for Science tech at @caltech. @YueLabCaltech going strong! 💪

@Young_AGI Or maybe COLM. For NeurIPS I feel like you would withdraw after the rebuttal

@lightetal pivoted to NeurIPS?

tfw you spend hours reviewing ICML papers and half of them withdraw before decisions 🥲

Artemis II launched yesterday, sending four astronauts on a journey around the Moon in an exciting milestone for space exploration. 10 days. 685,000 miles. Up to 25,000 mph. 🚀 Behind every one of those numbers is software that has to be flawless. We explored what it takes for AI agents to build software at that level of rigor. When our agents translated a mission-critical space math library from C to Safe Rust, they caught subtle errors in the test suite that expert reviewers had missed for years. This is the first crewed trip to the Moon since 1972 — hope their flight skills aren't too Rust-y 😉

@sukh_saroy You can actually get very far with just prompting. In addition to asking the model to generate 5 different responses, we also found that perturbing the prompt helps a lot. arxiv.org/pdf/2411.05010

🚨Breaking: Stanford researchers built a new prompting technique! By adding ~20 words to a prompt, it: - boosts LLM's creativity by 1.6-2x - raises human-rated diversity by 25.7% - beats fine-tuned model without any retraining - restores 66.8% of LLM's lost creativity after alignment Let's understand why and how it works: Post-training alignment methods like RLHF make LLMs helpful and safe, but they unintentionally cause mode collapse. This is where the model favors a narrow set of predictable responses. This happens because of typicality bias in human preference data: When annotators rate LLM responses, they naturally prefer answers that are familiar, easy to read, and predictable. The reward model then learns to boost these "safe" responses, aggressively sharpening the probability distribution and killing creative output. But here's the interesting part: The diverse, creative model isn't gone. After alignment, the LLM still has two personalities. The original pre-trained model with rich possibilities, and the safety-focused aligned model. Verbalized Sampling (VS) is a training-free prompting strategy that recovers the diverse distribution learned during pre-training. The idea is simple: Instead of prompting "Tell me a joke" (which triggers the aligned personality), you prompt: "Generate 5 responses with their corresponding probabilities. Tell me a joke." By asking for a distribution instead of a single instance, you force the model to tap into its full pre-trained knowledge rather than defaulting to the most reinforced answer. Results show verbalized sampling enhances diversity by 1.6-2.1x over direct prompting while maintaining or improving quality. Variants like VS-based Chain-of-Thought and VS-based Multi push diversity even further. You can find the paper link in the next tweet. 👉 Over to you: What other methods can be used to improve LLM diversity?

@ZamAlex8 😭

Does Actor-Curator also work on larger models? YES! Qwen-7B on Countdown:

@kpb_in_acad Yeah np! It’s a good work

@lightetal Thanks, Jonathan, for reading and sharing my work! Means a lot coming from you!:)

Great work by Kishan! Highly recommend a read.

@DimitrisPapail @yisongyue It’s honestly liberating to be able to spend more time exploring ideas rather than bug hunting

That is, choose to be bottlenecked by ambition not technical grunt work

@lateinteraction So true. LLM-based optimization has been discussed in research for a while now

I find it disappointing to see how much progress comes daily from late interaction, DSPy/GEPA, and RLMs but our slow industry only catches up after a lab person slaps a name like “autoresearch” or “deep research” on it lol. The future is already here, just not equally distributed

recursive language models at work!

One thing we learned: Generating more samples ≠ exploring more. Most samples are near-duplicates More compute often just means more redundancy The real bottleneck isn’t compute—it’s diversity. Compute without diversity is wasted. codespace-optimization.github.io

@0xJuliechen Happy bday Julie!

Turning 26. 🎂 my 2nd year in sf. 3rd year in the US. 20+ friends came to my bday party. feeling peaceful and beloved :) I've written a birthday recap every year since 22, each one from a different city: seoul, singapore, shanghai, philly. this year, I'm writing it in sf - the place i decide to lock in and grow my roots. be bold. be kind. the best is yet to come 🩵

@m1nj12 Maybe if the community focuses more on soundness rather than novelty that could help alleviate these problems/pressures?

Reviewing for ICML and it’s extremely underwhelming. One adds a component to existing framework but write like they proposed the entire thing. Another describe their method in a very misleading way (in my opinion, intentionally, to overstate the novelty/utility). :((

Backprop is dead for post-training?

Our ACTOR-CURATOR project page is now live: actor-curator.github.io We added a fun little animation of the RL post-training loop. You can play around with it and watch the curriculum evolve as the actor and curator learn together. Enjoy!

@amritsinghbedi3 @anas__barakat Thanks! This is very cool!

@lightetal Nice work! We analyzed how post-training performance is affected by prompt mixing through the concept of Prompt Interference. arxiv.org/abs/2602.21189 @anas__barakat

Post-training LLMs is like mixing a cocktail: Too much easy data → no learning Too much hard data → instability Wrong balance → collapse And today, we mix it by hand. What if the data mixture could be learned instead of hand-tuned? arxiv.org/abs/2602.20532 🧵👇