Researchers have mapped how on-policy distillation moves through a model's parameter space - and it does not look like fine-tuning or reinforcement learning.
On-policy distillation (OPD) is a training technique where a smaller language model generates its own sample outputs while learning to imitate a more capable teacher model. A new paper compares OPD's behavior in parameter space against supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR), using a suite of weight-space diagnostics. The researchers find OPD affects fewer weights than SFT and avoids dominant weight directions more aggressively, while remaining less constrained than RLVR. They also document what they call "subspace locking": early in training, OPD updates narrow rapidly into a low-dimensional channel, and that channel proves sufficient - models confined to it match full OPD performance, while the same constraint collapses SFT.
OPD is widely treated as a shortcut to reasoning capability - simpler than RLVR, more powerful than plain fine-tuning - and that framing has made it popular with teams trying to build smaller reasoning models on a budget. This paper argues it is not a point on that spectrum but a different regime with its own geometry, which means intuitions borrowed from SFT or RLVR may not transfer cleanly. For the practitioners who expanded distillation use after smaller distilled models proved competitive with much larger models trained from scratch, that is a meaningful recalibration.
The subspace locking result is the most operationally interesting: if OPD commits early to a narrow update channel, there may be a case for shorter training runs than teams currently assume - though the paper surfaces the observation rather than resolves it.