A new training method teaches AI models token by token, not answer by answer.
The paper "Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning" (arXiv:2609.15404) proposes VG-OPD, a way to fold several specialist "teacher" models into one student model. Standard multi-teacher distillation hands each training prompt to a single domain expert and treats every token in that expert's response as equally important. The paper's authors found that assumption doesn't hold: a teacher's useful signal is concentrated in a handful of tokens, not spread evenly across a response. VG-OPD instead runs a verifier that checks each teacher's answer against specific criteria, then only lets that teacher's guidance shape the exact tokens where it demonstrably helps.
Tested on 4B and 8B parameter student models across seven scientific reasoning benchmarks, VG-OPD ranked first on five of them, with the largest gains on knowledge-intensive science questions. The paper's own ablations show those gains come from placing supervision on the right tokens, not from adding more teachers or more distillation loss. Misplacing the same supervision budget, the authors report, was the single most damaging change they tested, worse than skipping distillation altogether.
That is a useful data point for anyone building many-teachers-one-student training pipelines. More experts and more distillation are not automatically better, and sloppy, unverified distillation can drag a model below the performance of plain reinforcement learning.