Sabitlenmiş Tweet
Training with more data = better LLMs, right? 🚨
False! Scaling language models by adding more pre-training data can decrease your performance after post-training!
Introducing "catastrophic overtraining." 🥁🧵+arXiv 👇
1/9
English
Jacob Springer
126 posts
Jacob Springer retweetledi
Had a great time working on this project exploring how to proactively prevent forgetting of capabilities during subsequent training! All credit goes to @lawrencefeng17 for leading it so skillfully!
Lawrence Feng@lawrencefeng17
1/ To retain post-training capabilities after further fine-tuning, mix that data into pretraining. The effect can be invisible until fine-tuning begins; early exposure may not help post-training performance, but it changes what persists. How a model learns a task matters.
English
Jacob Springer retweetledi
1/ To retain post-training capabilities after further fine-tuning, mix that data into pretraining.
The effect can be invisible until fine-tuning begins; early exposure may not help post-training performance, but it changes what persists.
How a model learns a task matters.
English
Jacob Springer retweetledi
It's one of the first lessons in ML: the model with the lowest train loss isn't the one that generalizes best. Pretraining made that easy to forget. You train for one epoch over trillions of tokens, there's no traditional overfitting, and pretrain loss starts to feel like the whole story.
Our paper argues it isn't. The lowest-loss model isn't the best starting point for post-training. An old sharp-vs-flat lesson, back in a new regime.
Ishaan Watts@IshaanWatts18
Spending billions to train the "best" base model? You might be optimizing the wrong thing! 🎯 We show that controlling sharpness during mid-training leads to over 35% less forgetting after fine-tuning / quantization... even when the base model itself gets worse. 🧵 Takeaways for pretraining: - Use SAM (Sharpness-Aware-Minimization) in the final steps (~10%) - Try much higher learning rates (yes, even ~10× larger) 1/9
English
I also hope our work helps the open source model development community pre-train better models that are easier to fine-tune; would love to see some of this implemented in Marin @percyliang, OLMo @natolambert, or SmolLM @eliebakouch
English
But I'm excited to see if we can do better. I would love to see a nanoGPT speedrun benchmark that evaluates models based on how well they can be post-trained. I suspect we'll learn that a lot of the optimization lessons we think we know end up being (at least subtly) wrong.
English
Just released a new pretraining paper with some interesting takeaways:
- sharpness minimization is important but it doesn’t show its benefit until *after* you post-train
- increase your learning rate!! (this is free!)
read Ishaan’s thread but I’ll also add my 2 cents below
1/n
Ishaan Watts@IshaanWatts18
Spending billions to train the "best" base model? You might be optimizing the wrong thing! 🎯 We show that controlling sharpness during mid-training leads to over 35% less forgetting after fine-tuning / quantization... even when the base model itself gets worse. 🧵 Takeaways for pretraining: - Use SAM (Sharpness-Aware-Minimization) in the final steps (~10%) - Try much higher learning rates (yes, even ~10× larger) 1/9
English
RT @IshaanWatts18: Obrigado Brazil! 🇧🇷 Had an incredible time at @iclr_conf talking about our work on pretraining optimization.
I also had…
English
Jacob Springer retweetledi
for data-constrained pre-training, synth data isn’t just benchmaxxing, it lowers loss on the real data distribution as we generate more tokens
for even better scaling, treat synth gens as forming one long 𝗺𝗲𝗴𝗮𝗱𝗼𝗰: 1.8x data efficiency with larger gains under more compute
English
Jacob Springer retweetledi
Models are typically specialized to new domains by finetuning on small, high-quality datasets.
We find that repeating the same dataset 10–50× starting from pretraining leads to substantially better downstream performance, in some cases outperforming larger models. 🧵
English
Jacob Springer retweetledi
Your LLM already knows the answer. Why is your embedding model still encoding the question?
🚨Introducing LLM2Vec-Gen: your frozen LLM generates the answer's embedding in a single forward pass — without ever generating the answer. Not only that, the frozen LLM can decode the embedding back into text.
🏆 SOTA self-supervised embeddings
🛡️ Free transfer of instruction-following, safety, and reasoning
GIF
English
Jacob Springer retweetledi
to improve fine-tuning data efficiency, replay generic pre-training data
not only does this reduce forgetting, it actually improves performance on the fine-tuning domain! especially when fine-tuning data is scarce in pre-training (w/ @percyliang)
English
the rank of llm representations / weights has recently been such a hot topic, with multiple papers arguing that rank is a good predictor of performance
it turns out, our paper shows it's mainly hyperparameters that determine the rank!
read Atharva's thread ↓
Atharva Kulkarni@athrvkk
Is the geometry of language model weights really predictive of performance?🔍 Our new work challenges the popular hypothesis that low rank unembedding matrix hurts LLM performance; and the answer is more complicated than you'd think! arxiv.org/abs/2602.20433 (1/8)
English
Jacob Springer retweetledi
🔭 We’re releasing Hodoscope: an open-source tool for unsupervised behavior discovery. It lets you visually explore and compare agent behaviors at scale.
It helped us discover a novel reward hacking vulnerability in Commit0 - with just a couple minutes of human effort.
English
Jacob Springer retweetledi
Are we done with new RL algorithms? Turns out we might have been optimizing the wrong objective.
Introducing MaxRL, a framework to bring maximum likelihood optimization to RL settings.
Paper + code + project website: zanette-labs.github.io/MaxRL/
🧵 1/n
English
Jacob Springer retweetledi
RL on LLMs inefficiently uses one scalar per rollout. But users regularly give much richer feedback: "make it formal," "step 3 is wrong."
Can we train LLMs on this human-AI interaction?
We introduce RL from Text Feedback, with 1) Self-Distillation; 2) Feedback Modeling (1/n) 🧵
English
Jacob Springer retweetledi
1/ We found that deep sequence models memorize atomic facts "geometrically" -- not as an associative lookup table as often imagined.
This opens up practical questions on reasoning/memory/discovery, and also poses a theoretical "memorization puzzle."
GIF
English
Ironically hard drive space is the limiting factor in a lot of my experiments, not GPUs.
(life of a hard-drive-poor phd student)
English
Whoever decided tokenizers should have a vocab size of ~128,000 (2^17) was clearly not an engineer because now I have to store tokens as uint32 instead of uint16. One bit less and it would use half the disk space.
English