Adithya Bhaskar

Language models that think, chat better. We used longCoT (w/ reward model) for RLHF instead of math, and it just works. Llama-3.1-8B-Instruct + 14K ex beats GPT-4o (!) on chat & creative writing, & even Claude-3.7-Sonnet (thinking) on AlpacaEval2 and WildBench! Read on. 🧵 1/8

The paper shows that making models think before answering makes them chat better. It introduces reinforcement learning with model rewarded thinking, RLMT, which makes the model write a private plan, then the final reply. A separate reward model, trained from human choices, scores each reply so higher scoring answers get reinforced. Training uses group relative policy optimization, GRPO, which samples several replies per prompt and pushes the model toward the better than average ones. They try 2 setups, a warm start that learns the format from teacher examples, and a zero setup that skips supervised fine tuning. The prompts come from real chat requests, not math or code, so the model practices everyday tasks. Across Llama-3.1-8B and Qwen-2.5-7B, the thinking versions beat non thinking baselines on chat and writing by about 3 to 8 points. One 8B model beats GPT-4o on chat and creative writing, and is close to Claude-3.7-Sonnet. Language_Models_that_Think_Chat… The thinking style shifts from rigid checklists to listing constraints, grouping themes, checking edge cases, and then refining. Gains depend on a strong reward judge and a chat heavy prompt mix, which pushes benefits beyond verifiable tasks like math and code. ---- Paper – arxiv. org/abs/2509.20357 Paper Title: "Language Models that Think, Chat Better"

Ever wonder why some AI chats feel robotic while others nail it? This new paper introduces a game-changer: Language Models that Think, Chat Better. They train AIs to "think" step-by-step before replying, crushing benchmarks. Mind blown? Let's dive in 👇