Why iterating in hidden space helps neural networks reason better

Neural networks can improve reasoning by iterating on a hidden state at test time rather than computing an answer once. This paper shows that this works because networks learn to create dynamical systems with stable fixed points—attractors—that pull toward correct solutions. Equilibrium Reasoners scale two ways: running more iterations (depth) and sampling multiple trajectories from different starting points (breadth). On Sudoku-Extreme, the approach reaches 99% accuracy versus 2.6% for single-pass models, with performance gains closely tied to convergence toward solution-aligned attractors.