A new training method aims supervision only at the exact moments a smaller AI model wanders off its teacher's script.
On-policy distillation trains a smaller "student" model on trajectories it generates itself, rather than ones the teacher wrote, because that better matches what the student will actually see at test time. The catch: a weak student can wander into text the teacher would rarely have produced, where the teacher's guidance is less reliable. A paper posted September 30, 2026 on arXiv ("SAKI: Maximal-Coupling-Routed Teacher Supervision for On-Policy Distillation," arXiv:2609.36601) proposes SAKI, which pairs a KL-constrained rollout with a technique called maximal coupling to flag, token by token, whether the student's choice lines up with the teacher's or diverges. Aligned tokens get standard reverse-KL supervision; divergent ones get corrected directly toward the teacher's top pick, and the odds a token needs correction equal the total variation distance between the two models' distributions, so one setting controls both how far the student can drift and how often heavier correction kicks in.
The authors also built a speculative verifier that keeps the same trajectory statistics while running rollouts 4.22x faster, which matters more for adoption than any accuracy bump, since distillation pipelines are often throughput-bound. On accuracy, the paper reports SAKI beats a matched teacher-guided baseline on Mean@8 (average accuracy across 8 sampled attempts per problem) and Pass@8 (whether at least one of 8 attempts succeeds) across seven math reasoning benchmarks, for both 1.7B- and 0.6B-parameter students - though the abstract does not publish the actual percentage-point deltas, so how large the win is remains an open question until the full tables surface.
Targeted, disagreement-triggered supervision is a sensible idea - most prior on-policy distillation work applied uniform supervision regardless of how far a student had drifted. But until independent tests replicate that 4.22x throughput claim and the missing numbers show up, this is a promising preprint, not a settled result.