Mimicking an AI model's internal representations turns out to be a flawed goal, and a new paper explains the geometry behind why.
Researchers studying knowledge distillation — the technique of training a smaller "student" model to copy a larger "teacher" — argue the field has been chasing the wrong target. A pretrained model's internal features have no fixed coordinates; they exist only up to a class of mathematically equivalent transformations. Matching those features directly is therefore ill-posed, like trying to copy a shadow instead of the object casting it. What actually transfers capability, the study finds, is matching the teacher's output function — its logits — not its hidden-layer geometry.
The finding matters because distillation is one of the primary tools for compressing large models into smaller, deployable ones. If the dominant approaches are matching the wrong thing, years of engineering effort may have produced students that look geometrically similar to their teachers but underperform on the capabilities that count. The paper offers a unifying framework that reframes feature matching, relational distillation, and alignment as special cases of the same underlying geometric problem.
The researchers validated the framework on Qwen2.5 and Llama-3.1, running a restoration study that recovered a corrupted model's geometric structure to near-perfect similarity (CKA score of approximately 0.99) while failing to recover its actual capability — clean evidence that geometry and function can come apart. A follow-up graft experiment found that corpus coverage overlap predicts whether transplanting representations succeeds, but is necessary rather than sufficient. In short: you can rebuild the shape of a mind without rebuilding what it knows how to do.
For a field that has treated feature matching as a reliable proxy for capability transfer, that distinction is worth sitting with.