A new method makes test-time reinforcement learning dramatically cheaper by adjusting a tiny sliver of a model's parameters instead of the whole thing.
Most test-time reinforcement learning, or TTRL, approaches update a large chunk of a model's parameters to sharpen its reasoning while it is being used, which works but is computationally heavy. Researchers built a leaner version called label-free bias-only TTRL that freezes the pretrained model and adjusts only around 100,000 bias parameters, a tiny fraction of a typical model's total, rewarding it with majority-vote pseudo-labels, meaning whatever answer the model gives most often across repeated attempts, instead of human-labeled data. On the MATH-500 benchmark, this approach hit 76.67% accuracy, slightly beating the researchers' own version trained with real labels, while touching 76,000 times fewer parameters than standard full-parameter test-time training. The same technique also improved results on the vision-language benchmarks MathVista and AI2D, the reasoning benchmark LogicVista, and the audio benchmark MMAU, and the learned adjustments carried over to 4,500 separate math problems the model had not trained on.
This matters because it decouples reasoning gains from two of the biggest costs in deploying adaptive AI systems, expensive full-model retraining and the need for labeled data. If a model can improve itself using only its own consensus answers and a sliver of its parameters, teams with limited compute or no labeled datasets could still capture real accuracy gains. The researchers also found that majority-vote reliability climbs as more attempts are sampled, a reminder that this approach leans on volume rather than any new source of ground truth.
The paper does not report training or inference latency, so whether this shortcut is fast enough to matter outside a benchmark table remains unproven.