A new decoding technique lets language models reason better at inference time without any retraining.
Researchers introduced Entropy-Gated Latent Recursion (EGLR), a method that re-applies a frozen model's top decoder layers at tokens where the model is most uncertain — measured by entropy in the next-token distribution. The key insight is that existing inference-time scaling methods only vary outputs through stochastic temperature sampling, which is one axis. EGLR adds a second, fully deterministic axis: how many layers get recursively re-applied, and for how long. Combining both axes turns a single-dimensional rollout pool into a grid of candidates at roughly the same per-rollout cost.
The practical gains are real enough to notice. On the MATH-500 benchmark using Qwen2.5-3B-Instruct, the joint approach reached 91.6% accuracy — 8.2 percentage points above temperature-only sampling and 10.4 points above layer-recursion alone. That gap matters because it confirms the two axes are catching genuinely different problem types, not just adding noise.
The method slots into existing pipelines — self-consistency voting, best-of-N selection, and reinforcement learning training via GRPO — without requiring any model weight updates. That makes it cheap to evaluate against whatever scaling trick a lab is already running. Whether it holds up across non-math tasks remains the open question, but as a training-free add-on, the bar for trying it is low.