Researchers have a new way to stop AI teachers from passing bad habits to their students.
When training smaller language models, a common approach called on-policy distillation has the student model generate its own outputs, then uses a larger teacher model to guide it token by token. The problem: teachers are not always right. A teacher can assign high confidence to a wrong answer, or low confidence to a correct answer that just looks different from what the teacher expected. A team at the University of Cyprus addressed this with RG-OPD — Reward-Gated On-Policy Distillation — which adds a verification layer. Instead of blindly following the teacher, the student checks whether a given trajectory is actually correct before accepting the teacher's guidance on it.
The stakes here go beyond academic benchmarks. As AI labs race to shrink models for on-device and low-cost deployment, distillation quality directly determines how much reasoning ability survives the compression. A method that cuts misleading teacher signals could mean the difference between a compact model that reasons reliably and one that confidently gets things wrong. RG-OPD improved over a standard reverse-KL distillation baseline by 2.9 points at typical generation lengths, and by 8.2 points over an untuned student in long-generation tasks.
The approach threads a needle that has tripped up prior work: dense token-level supervision is useful, but only when the teacher is trustworthy — and now there is a gate. Whether those benchmark gains hold across domains beyond reasoning and coding remains the real test.