A small tweak to how AI models learn from their own mistakes is quietly improving multi-turn training results.
Researchers behind SMOPD (Selective Masking for On-Policy Distillation) tackled a specific problem: when a student model messes up partway through a multi-turn conversation, every later turn gets trained on that mistake, and standard distillation wastes effort on tokens that do not actually correct the error. SMOPD instead ranks each token in a generated reply by the student model's own uncertainty, or entropy, and strips the lowest-entropy 20% out of the distillation loss, leaving final-answer and preservation losses untouched. The method adds no new parameters and costs nothing extra at inference time. Tested on the LiC benchmark across Qwen3 models at 1.7B, 4B, and 8B parameters, it improved SHARDED-view accuracy by 1.0 to 2.5 percentage points, and a repeated multi-seed run on the 4B model showed a statistically significant 1.7 point average gain.
The more interesting finding is what did not work: scaling the loss by a proxy for final-answer correctness, a coarser trajectory-level signal, actually hurt performance at 1.7B parameters by 4 points and only helped inconsistently once combined with masking. That is a useful data point for anyone building distillation pipelines: token-level uncertainty looks like a sturdier training signal than outcome-based scoring, at least in this dirty-history setting.
Still, this is one benchmark, one model family, and mostly single-seed results, so treat the exact percentages as suggestive rather than settled. The authors themselves leave causal explanations and broader validation for later work.