A new paper argues that a popular way of training smaller AI reasoning models has been learning some of the wrong lessons, and it shows a fix that throws out roughly a third to half of the training signal without hurting performance.
On-policy distillation is a common way to train a smaller "student" AI model: it learns from the token-by-token gap between its own outputs and a stronger "teacher" model's outputs on the same problem. The researchers found that gap does not purely reflect the teacher's edge in ability. Some of it is just the teacher's own inconsistency, and a variant called privileged distillation, which feeds the teacher extra information, makes that inconsistency worse and gets copied into the student anyway. Their method, called Calibrated On-Policy Distillation, probes the teacher with positive and negative interventions to map out where it is being inconsistent with itself, then strips that region out before training the student.
Distillation is how most cheap, fast AI models inherit reasoning ability from giant, expensive ones, so cleaner training signal is not a minor detail. On math reasoning benchmarks, the calibrated version kept only 52 to 65 percent of the original signal and still beat standard on-policy distillation across model sizes, meaning a large chunk of what teams have been training on was arguably wasted motion.
It is a useful reminder that in AI training, more signal is not automatically better signal, sometimes the teacher is just talking to itself.