AI agents are worse at remembering than the field has bothered to measure.
Researchers have released MemoryAgentBench, a benchmark built specifically to test memory in large language model agents across four competencies: accurate retrieval, test-time learning, long-range understanding, and selective forgetting. The work draws on cognitive and memory science theory, then converts existing long-context datasets into multi-turn formats that simulate how agents actually accumulate information over a conversation — not just how they handle a static wall of text. The team tested a range of systems, from basic retrieval-augmented generation setups to agents with dedicated external memory modules. None of them mastered all four skills.
The gap matters because most existing benchmarks measure reasoning, planning, and task execution — things that look impressive in demos. Memory, meaning how an agent updates what it knows and discards what it no longer needs, has been quietly left ungraded. A model that aces logic puzzles but can't track a fact introduced ten turns ago is a lot less useful than its leaderboard position suggests.
The selective forgetting criterion is the most underappreciated piece here. Remembering everything is not the same as remembering well — an agent that can't prune stale or contradictory information will compound errors across a long session, not just fail a single question. That's a harder problem than recall alone, and it's the kind that tends to stay hidden until a product ships.