Safal Shrestha

🚨New paper! "Layer Importance for Mathematical Reasoning is Forged in Pre-Training and Invariant after Post-Training" We found something surprising about how LLMs get better at math: the critical layers for mathematical reasoning are forged during pre-training and stay remarkably stable afterward, no matter what post-training you do. We studied base models and their instruction-tuned/RL/distilled variants using layer-wise ablation. Question: Do math gains come from major architectural changes or subtle adjustments that preserve the original structure? We found that math reasoning relies on a small set of critical layers. Ablating these layers results in an 80% drop in math accuracy, while factual recall tasks show much smaller drops. These critical layers remain invariant across post-training methods. The layers that matter for math are identified during pre-training and stay locked in place. It seems that post-training just tunes them and doesn't restructure them. We also measured what happens to token representations near these critical layers using NMI. Tokens drift from syntactic clusters toward representations that are more semantically useful for downstream mathematical tasks. Shoutout to all the great people who did this project! @NepalAadim, who is the lead author, and on the PhD job market (he's great, you should hire him!), will present it at the BlackboxNLP EMNLP workshop on Sunday! Paper: arxiv.org/abs/2506.22638 @jalalnaghiyev06 @MinwuKim3, Anubhav Shrestha, @saffffal, Keith Ross

@VioletNPeng served as the Program Co-Chair for #EMNLP25, one of the largest NLP conferences, which received over 8,000 submissions and drew 6,000 participants.