AI/ ai-memory · llm-research · conversational-ai · benchmarks

AI Memory System Tracks Facts Across Group Chats

A new research method verifies and updates facts in group conversations, beating prior systems by over 20 points on memory benchmarks.

Researchers have built a memory system for AI chatbots that can track facts across sprawling, multi-person conversations without getting confused about who said what or forgetting when something changed.

The system, called EGMEMORY, treats conversation memory as a searchable state machine. It keeps raw message evidence separate from a running summary of "what's currently true," and updates that summary only after proposing a change and checking it against the evidence. When answering a question later, it works backward from the query, using the conversation's structure and a mix of keyword and semantic matching to find the right facts. No specialized training was needed - it runs through prompting and tool use on top of an existing model.

The results are the real story here: 68.2% on GroupMemBench and 77.9% on EverMemBench, beating the best baselines by 22.7 and 21.4 percentage points. That's not an incremental bump. It suggests most current chatbot memory tools are still bad at the specific problem of untangling who-said-what in group settings, and that separating "evidence" from "current state" is a bigger lever than better retrieval alone. The system also scored 73.6% on LoCoMo, a two-person benchmark, showing the approach isn't just tuned for group chat.

Every assistant that claims to "remember" your conversations is quietly betting on architecture like this. The code isn't out yet, so the claims are still just numbers on a leaderboard.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →