Training a reasoning model on its own mistakes just got a bit more principled.
Researchers have proposed Privileged Hidden Flow (PHF), an extension to on-policy self-distillation (OPSD) — a training method where a model learns from rollouts it generates itself, guided by a teacher that can see verified reference answers. The catch with existing OPSD approaches: they only supervise what the model outputs, not the internal computation that produced it. PHF adds a second layer of supervision by tracking how the teacher model's internal states move through a sequence — aligning the direction and geometry of those transitions rather than forcing a token-by-token match. It's a subtle but meaningful distinction: you're teaching the student model to reason along similar paths, not just arrive at similar answers.
The method posted consistent gains across three sizes of the Qwen3 model family — roughly 2.2, 1.5, and 1.7 points on the Average@12 benchmark for the 1.7B, 4B, and 8B variants respectively — all under an identical 100-step training schedule. Those numbers aren't headline-grabbing, but incremental and reproducible gains in model training efficiency matter more than flashy jumps that don't generalize. PHF is also designed with some useful invariance properties: the core transport objective doesn't break when trajectories share a common offset, and the geometry term holds up under orthogonal transformations.
The broader context here is a quiet arms race in making smaller models reason better without scaling compute. Techniques like PHF sit in a growing toolkit — alongside reinforcement learning from human feedback and process reward models — that try to extract more signal from training runs rather than simply training longer or larger. Whether PHF's hidden-flow alignment survives contact with larger models and harder benchmarks remains to be seen.