A new training method squeezes useful lessons out of AI teacher models even when those teachers get the answer wrong.
The technique targets offline on-policy distillation, a way of shrinking large AI models into smaller, cheaper ones by having a "student" model learn from a "teacher" model's example trajectories, collected once and reused throughout training. The catch: when a teacher fails a problem, that failure was assumed to poison the whole trajectory, so researchers built a workaround. Their method trains on the problems the teacher solved correctly, then checks how much that success-only training shifts the likelihood of each token appearing in the failed trajectories - a signal for which parts of a bad trajectory are still worth learning from. Tested on math reasoning and code generation, it improved on a standard offline baseline by up to 2.7 percentage points and matched or beat online distillation methods that re-run the teacher live.
The efficiency angle is the real story. Online distillation methods needed 3 GPUs and 36 to 48 GPU hours in the paper's comparisons; this offline approach hit comparable or better results with 2 GPUs and about 22 GPU hours, without generating any new teacher data. For teams distilling frontier models into smaller deployable ones, that is a meaningful compute discount, not a rounding error.
It is one arXiv preprint, not a peer-reviewed result, and the gains are benchmark-specific to math and code tasks. Whether the trick holds up on messier, more open-ended teacher outputs is the next question worth asking.