A new training method promises to make AI agent distillation at least four times faster per step, without sacrificing accuracy.
Researchers studying on-policy distillation — the process of training a smaller "student" model to mimic a larger "teacher" model on agentic tasks — identified a core inefficiency: every training update required the student to re-run through its environment and query the teacher from scratch. Their proposed fix, Replayed-Prefix On-Policy Distillation (ReOPD), sidesteps this by recycling pre-collected teacher trajectories as reusable starting points. The student only acts at selected steps; the teacher provides supervision without touching the live environment again. A step-decaying sampling schedule prioritizes earlier, more reliable parts of those trajectories to avoid feeding the student on histories where the teacher's own guidance becomes shaky.
The efficiency gap matters because agentic AI systems — models that use tools, browse the web, or write and execute code — are expensive to train relative to standard language models. Any method that converts live environment interaction into an offline resource could meaningfully lower the bar for labs that lack the compute to run continuous rollouts. The paper reports ReOPD matches or beats standard on-policy distillation accuracy across math-reasoning and search tasks at multiple model scales, while using zero tool calls during student training.
Distillation is increasingly the path smaller labs take to get competitive models without training from scratch — so a 4x speedup in the agentic slice of that pipeline is the kind of unglamorous infrastructure win that tends to compound quietly.