A study published in PNAS Nexus this month finds that transformer attention mechanisms are structurally deficient in a specific cognitive capacity: executive control.
In human cognition, executive control is the brain's ability to direct and suppress attention — prioritizing task-relevant signals while filtering out noise. It is a top-down process, driven by goals rather than by incoming data. Transformer attention works the other way: it is data-driven, computing relevance across all input tokens simultaneously with no inherent mechanism to downweight distractors or stay anchored to a goal. The paper argues this is not an implementation detail but a fundamental architectural gap — the attention mechanism attends to everything, which is another way of saying it focuses on nothing.
The implications are practical. Practitioners have long observed that large language models perform worse when irrelevant information is embedded in context, degrade under prompt injection, and drift off task as inputs grow long or ambiguous. This study offers a mechanistic account for those failure modes — one that goes beyond "the model needs more training" and points to a structural property the architecture does not currently have. That distinction matters for anyone deciding whether scaling or fine-tuning alone can close the gap.
The word "attention" was always borrowed from cognitive science as a metaphor. This paper suggests the metaphor was more aspirational than accurate.