A new paper picks apart why self-distillation training so often makes models weirdly terser or weirdly longer-winded, and proposes a fix that keeps the correctness signal without the side effects.
Researchers tested two ways of training a model against a privileged teacher version of itself: pulling the model toward the teacher's answers (attractive) and pushing it away from the teacher's answers (repulsive). Attraction made models more confident and terse, cutting down on exploratory reasoning. Repulsion did the opposite - responses got longer, sometimes triggered unintended switches into the model's internal thinking mode, and eventually became unstable. The team then combined both signals into a single contrastive objective: attract toward a teacher conditioned on the correct solution, repel from one conditioned on an incorrect solution.
That combination cancels out the behavioral side effects each approach introduces on its own, leaving a training signal that more directly tracks whether an answer is right rather than how verbose or hedgy the model sounds. Tested across non-thinking, instruct-only, and already-thinking models, the contrastive method improved reasoning performance while keeping response lengths stable - the two things that trip up cruder distillation setups.
It's a reminder that in reasoning-model training, correctness and style are easy to conflate and hard to pull apart - and that untangling them, rather than adding more compute, is where a lot of the real gains are hiding.