A new distillation technique lets compact AI models absorb not just what bigger video-reasoning models say, but how they think to get there.
Researchers propose Latent-OPD, which extends on-policy distillation for video reasoning models by matching internal hidden states, not just output tokens, at the end of each reasoning trajectory. They add a progressive teacher-lookahead strategy that lines up middle-to-late student layers with increasingly deeper teacher layers as training goes on. Tested across six video reasoning benchmarks, the method beat standard output-only distillation. The gains were largest on videos with few frames, long runtimes, or tasks that require piecing together evidence scattered across many scenes.
Video reasoning models are expensive because they have to chew through huge amounts of visual data, and shrinking them has mostly meant training a small model to copy a big one's final answers. Latent-OPD's bet is that a student model which mimics a teacher's internal reasoning state, not just its conclusion, generalizes better when frame budgets are tight - exactly when cheaper models matter most in production.
Six benchmarks from one research team is not the same as proof this holds up at deployment scale, and trajectory-level latent alignment still has to survive contact with real-world compute budgets before it's more than a promising paper.