A paper on on-policy distillation finds that a significant chunk of training effort is being wasted on tokens that never actually help a model think better.
Researchers studying how smaller AI models learn from larger "teacher" models identified a category they call Rock Tokens — tokens that persistently show high loss even after training appears to have plateaued. These tokens can make up as much as 18% of a model's generated output. The paradox: despite consuming a disproportionately large share of gradient updates (the mathematical nudges that drive learning), Rock Tokens don't budge. They resist correction and, when tested through causal intervention, contribute essentially nothing to the model's actual reasoning ability.
This matters because on-policy distillation is one of the primary methods labs use to compress large, expensive models into smaller, deployable ones. If nearly a fifth of the training signal is being burned on structural filler the student model neither needs nor can absorb, that's wasted compute at scale — and potentially a ceiling on how good distilled models can get. The authors argue that skipping or down-weighting these tokens can meaningfully streamline the alignment process.
The finding rhymes with parallel work in reinforcement learning that identified similarly lopsided token dynamics, suggesting that blanket, uniform token weighting may be a bigger inefficiency than the field has acknowledged.