“Solve the Loop: Attractor Models for Language and Reasoning” Looped Transformers can refine their thoughts internally, but they are usually unstable and tied to a fixed number of loops. So this paper turned recurrence into a fixed-point problem, where a Transformer first makes an output-embedding guess, then an attractor module refines it until convergence. This makes iterative reasoning trainable with constant memory, adaptive depth, and less compute. The surprising part is equilibrium internalization because after training, the model learns to start near the fixed point, so the solver can almost disappear at inference. In their experiment, a 770M Attractor Model beats a 1.3B Transformer trained on twice the tokens, and a 27M model gets 91.4% on Sudoku-Extreme and 93.1% on Maze-Hard.
