A new framework measuring how retrieval accuracy degrades under competing associations shows RAG systems struggling more than human memory — with one neuroscience-inspired exception.
Researchers applied signal detection theory — a statistical tool from sensory psychology — to both human episodic memory and retrieval-augmented generation in matched experiments. Both systems showed logarithmic accuracy drops as the number of stored associations per concept grew, a pattern psychologists call the fan effect. But the sensitivity scores split: human memory scored 0.41 on a key interference metric, standard dense-passage retrieval scored 0.67, and HippoRAG — a RAG variant designed to mimic hippocampal memory organization — landed at 0.44. The framework was validated against 112 human participants and confirmed through parameter recovery.
Most RAG benchmarks test clean retrieval: find the right chunk when the answer is unambiguous. This study targets the messier, more realistic case — what happens when a knowledge base is dense with semantically similar entries, precisely the environment enterprise deployments operate in. The gap between human memory and AI retrieval under noise has real stakes: it predicts where today's RAG pipelines silently fail.
The authors offer six falsifiable predictions and wisely stop short of prescribing fixes, noting that the causal role of candidate mechanisms remains to be established — which is the honest move, even if builders waiting on a HippoRAG upgrade will still be waiting.