A new residual connection lets a tiny AI model keep reasoning accurately through more than 20,000 loop iterations, instead of forgetting its own progress.
The research, posted to arXiv, addresses looped transformers - models that reuse the same block of layers over and over rather than stacking new ones for more depth. The authors found a flaw: as loop counts climb, noisy updates can overwrite correct intermediate reasoning, sometimes undoing a problem the model had already solved, and later loops struggle to recover what was lost. Their proposed fix, InfiLoop, adds a loop-native residual connection that learns which past computations to keep and how much of each new update to accept, blending content-based weighting with a learned decay rate. A streaming version of that mechanism keeps memory use flat as loop count grows, and a 7M-parameter model built on it reached 97.9% exact accuracy on the Sudoku-Extreme benchmark and 13.6% pass@2 on ARC-AGI-2, with accuracy on Sudoku-Extreme still climbing past 20,000 effective reasoning steps.
Most progress in reasoning models comes from training bigger networks or generating longer chains of text at inference time. This result suggests a third lever: architectural plumbing that lets a small model squeeze far more useful computation out of each extra loop, rather than degrading under it. A 7M-parameter model beating larger recursive architectures on these tasks is a reminder that efficient reasoning may be more about how state is preserved than how many parameters are thrown at the problem.
Whether that holds up outside tidy benchmarks like Sudoku grids and ARC puzzles is the open question - real-world reasoning tends to be messier than either.