Researchers set out to prove that transformer models can internally 'reject' wrong answers. The geometry says something murkier.
The study forced seven decoder-only transformer models to process both the correct and incorrect single-token completions of factual questions. The hidden-state vectors for the two paths stayed close in magnitude but rotated apart as information moved through mid-depth layers, before late layers produced a lopsided result. That lopsidedness cuts against the 'rejection' story: in the incorrect run, the logit lens showed roughly 11.5 times more probability mass on the wrong token than the right one, meaning the model was heavily loaded toward the bad answer, not away from it. The pattern held across six architectures (Llama, Mistral, Gemma, and StableLM) spanning 1B to 13B parameters; Qwen2 1.5B was flat under the same protocol, possibly a tokenizer artifact.
Whether the late-layer asymmetry reflects genuine internal rejection or the model simply conforming to the token it was forced to carry is a question the study cannot resolve. The researchers note only a random-token control can settle it, and that control was not run. That gap matters: the mechanistic interpretability field has staked a lot on the idea that internal representations track truth, not just surface token statistics.
Activation patching, run on a subset of the models, found no single layer responsible for the asymmetry, fitting a distributed-across-layers account rather than a clean circuit. The field has been hoping for neater module boundaries inside large transformers. It is not finding them.