The Grumbling Loner

A follow-up study on Apple's "Illusion of Thinking" Paper is published now. Shows the same models succeed once the format lets them give compressed answers, proving the earlier collapse was a measurement artifact. Token limits, not logic, froze the models. Collapse vanished once the puzzles fit the context window. So Models failed the rubric, not the reasoning. ⚙️ The Core Concepts Large Reasoning Models add chain-of-thought tokens and self-checks on top of standard language models. The Illusion of Thinking paper pushed them through four controlled puzzles, steadily raising complexity to track how accuracy and token use scale. The authors saw accuracy plunge to zero and reasoned that thinking itself had hit a hard limit. 📊 Puzzle-Driven Evaluation Tower of Hanoi forced models to print every move; River Crossing demanded safe boat trips under strict capacity. Because a solution for forty-plus moves already eats thousands of tokens, the move-by-move format made token budgets explode long before reasoning broke. 🔎 Why Collapse Appeared The comment paper pinpoints three test artifacts: token budgets were exceeded, evaluation scripts flagged deliberate truncation as failure, and some River Crossing instances were mathematically unsolvable yet still graded. Together these artifacts masqueraded as cognitive limits. ✅ Fixing the Test When researchers asked the same models to output a compact Lua function that generates the Hanoi solution, models solved fifteen-disk cases in under five thousand tokens with high accuracy, overturning the zero-score narrative.