AI citations may be less trustworthy than they appear.
Researchers studying the Llama-3.1-8B-Instruct model found that when a large language model adds an inline citation to an answer, it is not consulting a tidy internal checklist. Instead, the decision emerges from a distributed, multi-stage cluster of attention heads and MLP layers the authors call an "attributional ensemble." Using a technique called activation patching on the PopQA dataset, they were able to map which components drive citation behavior — and then deliberately amplify or suppress them. Turning those components up repaired more than 90 percent of missed citations; turning them down eliminated 69 percent of spurious ones, without hurting answer accuracy.
That finding matters because retrieval-augmented generation is one of the main strategies the industry has adopted to make AI outputs verifiable. The implicit promise is that a citation means the model actually used that source. This research suggests the promise is not always kept — the model's visible reasoning and its internal computation can come apart, meaning a citation can appear credible without genuinely reflecting what the model drew on.
Gains on a harder, multi-document benchmark called HotpotQA were more modest, which keeps this from being a clean solution. But the same component set still nudged citation rates in the intended direction, suggesting the mechanism is general enough to matter beyond a single test set. The broader takeaway: inline citations in AI-generated text are a UX feature as much as an epistemic one, and building trust around them requires understanding the machinery, not just counting them.