A new benchmark says AI agents remember differently depending on how their memory is built, and no single approach wins everything.
Researchers tested four memory backends for AI agents on MemoryArena, a benchmark designed to check whether memory holds up across interdependent, multi-session tasks rather than simple recall. The four systems were MemoryLake, a structured multi-track memory backend; Mem0, a separate memory system; a vector-search setup built on OpenAI's text-embedding-3-small embeddings; and a long-context control that skips retrieval and just feeds everything into the prompt. All four ran on the same agent framework, the same requested gpt-5-mini model alias, and the same task sets across five domains: mathematics, physics, progressive retrieval, travel planning, and web shopping. MemoryLake posted the highest success rates in math (9 of 40), physics (12 of 20), and progressive retrieval (4 of 20), while every system scored zero on travel planning and web shopping produced just one bundle-level success, credited to the long-context control.
The headline number - a 20.5% average success rate for MemoryLake versus 13.6% for the best comparator - looks like a clean win, but the researchers flag small sample sizes, overlapping confidence intervals, and no paired significance testing. That caveat matters more than the leaderboard: no backend dominated every task, and travel planning stumped all four systems equally, which points to some benchmark domains being genuinely hard for current memory designs rather than one architecture being broken.
It is a rare benchmark paper that undersells its own result instead of overselling it.