Teaching a smaller AI model to reason like a larger one is harder than it looks.
A new paper from researchers proposes Trajectory-aware On-Policy Distillation (TOPD), a refinement of a popular training technique called on-policy distillation (OPD). The standard approach trains a student model by having it generate its own responses while a teacher model supervises — but the learning signal stays at the individual token level. The researchers found that roughly 30% of the tokens flagged as problematic are actually surface-level word-choice differences, not genuine reasoning errors. Fixing those wastes training signal and leaves real problems unaddressed.
TOPD addresses this by looking ahead in the response sequence rather than treating each token in isolation. Instead of patching a single flagged token, it spreads corrective guidance across several future tokens — closer to how reasoning actually fails, which tends to be a gradual drift rather than a single wrong word. That shift in focus pushed average accuracy from 47.8% to 52.2% across benchmarks, with gains on the AIME math competitions jumping from 60.0% to 63.3% on AIME24 and from 46.7% to 53.3% on AIME25.
The result matters because knowledge distillation — training cheaper, smaller models to approximate expensive, larger ones — is one of the main ways the industry keeps inference costs manageable. If the training signal itself is noisy, the efficiency gains erode. A more precise signal means smaller models can punch closer to their weight class without extra compute.
This is incremental progress, not a leap — but in a field where benchmark gains often come from scaling up rather than training smarter, a method that squeezes more from existing architectures is worth watching.