Azwar Abdulsalam

5/ TL;DR: the base model provides weak ingredients; RL assembles them into reliable higher-level strategies. 📄 arxiv.org/abs/2607.07646 Accepted at the Compositional Learning Workshop @ ICML 2026

New paper w/ @SaxeLab & Nishil Patel! 🧵 Does RL post-training teach models anything new, or just amplify skills already in the base model? We built a fully auditable testbed to settle it — and caught RL composing new strategies in the act.

4/ Rejection fine-tuning? Improves early, then plateaus. The difference isn't exploration volume — it's selectivity. RFT churns out shortcut-y, often invalid rewrites. RL concentrates its exploration into valid, reusable structure.

3/ The result: with only a final-answer reward, RL solves held-out problems the base model basically never solves. And we can see how: it first sharpens primitive skills, then builds composed procedures out of them, and reuses them as a stable toolkit.

2/ Why this is hard to prove in LLMs: you never know what was in pretraining. Our fix — a rewrite-grammar world where the pretraining distribution is fully known and every rewrite the model makes can be verified.

Paper link: #discussion" target="_blank" rel="nofollow noopener">openreview.net/forum?id=Pz9sd…

5/5 The broader takeaway: For world models, the right primitive may not be “predict the next observation.” It may be: learn a latent state, learn the motion that transforms it, and roll out by composing motions.

1/5 Excited to share LaMo: A Latent Motion World Model for Long-Horizon Prediction, to be presented at the ICLR 2026 Workshop on World Models. LaMo predicts compact latent motion rather than the next dense latent state.