Researchers have found a more precise way to locate which parts of a large language model do the work of understanding context, not just copying it.
The paper introduces Logit-Contribution Scoring, or LOCOS, a method for identifying so-called "non-literal retrieval heads" - the attention heads that synthesize meaning from a document rather than extracting exact strings. Existing detection methods only catch heads that copy tokens verbatim, missing the ones that actually reason. LOCOS instead measures how each head's output pushes the model toward the right answer token, contrasting relevant and irrelevant source positions in a single forward pass. The researchers tested it across three model families: Qwen3, Gemma-3, and OLMo-3.1.
The results clarify something that has been murky in AI interpretability research: where, mechanistically, long-context comprehension actually lives inside a model. On Qwen3-8B, disabling just 50 heads identified by LOCOS drives a standard retrieval score from 0.401 to zero - while the best prior method still leaves the model scoring 0.292 with the same heads removed. That is not a marginal improvement. And crucially, the ablated heads appear specific to retrieval: arithmetic and factual recall tasks stay at baseline, suggesting the method is carving the model at real joints rather than blunt ones.
Interpretability research has accelerated as AI labs face pressure to explain model behavior to regulators and enterprise buyers, but most techniques work at a coarse level. LOCOS points toward finer-grained circuit-level tools - which matters most when the question is not just "did the model get it right" but "what inside the model got it right, and can we trust it."