Researchers have a new fix for a stubborn LLM failure mode: reasoning that quietly falls apart the longer it runs.
A paper posted to arXiv describes SAGE (Structural Admissibility-Guided Exploration), a framework aimed at long-horizon reasoning tasks where rewards are sparse and feedback is rare. The authors identify two specific failure patterns: an exploration bias that pulls models toward reasoning branches that look plausible step-by-step but are structurally unstable, and a compounding bias where tiny early deviations snowball across many steps and bury the correct answer. SAGE tackles both with two techniques bolted together: algebraic sparsification, which narrows candidate reasoning steps down to a smaller, well-structured set, and hyperbolic structural guidance, which maps reasoning states into a curved geometric space to give the model steadier signal at every depth. The team tested it across 12 benchmarks and 7 model families.
The headline result is an up-to-8-fold improvement on the Andrews-Curtis problem, a genuinely hard open math task used as a stress test for long-horizon reasoning. That matters because most LLM reasoning fixes - better prompting, more chain-of-thought, bigger context windows - treat the symptom, not the structural reason models drift off course over many steps. SAGE is instead a bet that the geometry of the reasoning space itself, not just the model doing the reasoning, is where the leverage is.
Whether that generalizes past benchmark tasks to messier real-world agent workflows is the open question, and the code is public for anyone who wants to check.