A new study pins down exactly which settings control how well an AI model learns from its own output.
Researchers ran 1,200 training experiments on two language models, Qwen2.5-7B and Ministral-3-3B, testing a technique called self-distillation with privileged context. Here, a model is shown a reference answer, then has to teach a context-free copy of itself, token by token, without letting that copy see the answer. Prior work bundled together three separate settings: whether practice examples come from the student or the more-informed teacher version, how closely the teacher tracks the student over time, and which direction a key statistical comparison runs. By testing every combination instead of a fixed mix, the team found that teacher-generated examples help most when a task contradicts what the model already believes, with no cost to retained skills, and that tightening the teacher-student link boosts learning only up to a point before it backfires.
This matters because it explains why earlier papers on self-distillation reached conflicting conclusions: they were each testing a different corner of the same three-dimensional space and mistaking a local result for a general rule. For anyone fine-tuning models in production, that is the difference between guessing at hyperparameters and having an actual map of the trade-offs.
It is a dense, unglamorous paper, but it is the kind engineers debugging a model's selective amnesia will actually reach for, not one built for a launch post.