A new study finds that most ways AI agents store long-term memory quietly break when you swap the underlying model - except one.
Researchers compared four memory formats using 48 synthetic histories run through two open-weight models under 10 billion parameters: keeping the full raw history for long-context reading, chunking it for retrieval-augmented generation (RAG), compressing it into natural-language notes, or normalizing it into a fixed-schema knowledge graph. After swapping the model that originally wrote the memory, knowledge-graph accuracy barely moved, changing by just 0.04 percentage points on average. Compressed notes were far less stable, swinging between a 9.91-point gain and a 13.28-point loss depending on which direction the swap went. RAG systems that only partially re-embedded old memories into new vector space recovered less than half the accuracy gain of a full re-embedding.
The diagnostics are the useful part. Eighty percent of the notes format's accuracy loss traces back to information dropped during the original compression step, not to anything that happens later - meaning the damage is baked in before a new model ever reads the memory. And when researchers tried to repair degraded notes without access to the original raw history, they hit the 90-percent recovery target in zero of 48 test cases.
For anyone shipping agents with persistent memory, this is a warning that summarizing chat history - the default approach in most agent frameworks - is also the most fragile one. Structured storage looks like more upfront engineering, but the study suggests it is the difference between a memory that survives your next model upgrade and one that quietly rewrites itself.