[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-study-finds-memory-benchmarks-confuse-retention-with-retrieval":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},9461,"study-finds-memory-benchmarks-confuse-retention-with-retrieval","Study Finds Memory Benchmarks Confuse Retention With Retrieval","A new benchmark shows most gains from smarter AI agent memory systems come from extra history access, not better retrieval.","Most of the hype around AI agents with \"better memory\" turns out to be about how much history they can see, not how smart their retrieval is.\n\nResearchers built a streaming-recall benchmark that separately tests retention and selection, running 300 seeded episodes per condition. When they held access to history fixed, giving an agent query-aware selection, meaning it chooses what to surface based on the question asked, improved recall of required facts by 15.5 percentage points. But in a mixed comparison that let both retention and access vary at once, the reported advantage ballooned to 68.7 points, and 53.2 of those points came purely from extra access to history rather than smarter selection. Under a fixed memory budget, query-aware, dense, and even oracle selection methods all hit the same retention ceiling, and every one of 319 observed failures traced back to facts being evicted from memory, not to bad ranking. Recall fell to zero once a fact receded far enough into the past. The same pattern held on the SQuAD benchmark, where dense retrieval beat simple keyword matching but retention stayed the real bottleneck.\n\nThat reframes a chunk of existing agent-memory research. Papers comparing methods with different history budgets are mostly measuring whose memory is bigger, not whose retrieval algorithm is sharper. For anyone building long-running agents with limited context windows, this suggests bounded retention policy, what gets thrown away and when, deserves as much engineering attention as the retrieval model sitting on top of it.\n\nIn other words, \"our agent remembers more\" is often a context-window claim wearing an algorithm's clothes.","[\"ai-agents\",\"memory-systems\",\"benchmarks\",\"retrieval\"]","2026-10-02T04:00:00.000Z","2026-10-02T20:12:57.053Z","2026-10-02T20:13:08.407Z","published",null,[],"ai",[26,27,28,29],"ai-agents","memory-systems","benchmarks","retrieval",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.00366",0,{"sections":36},[37,40,44,48,53,57,61,66,71,76,81,86,91,96],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",6058,{"name":41,"slug":42,"count":43,"latest_published_at":18},"Security","security",848,{"name":45,"slug":46,"count":47,"latest_published_at":18},"Policy","policy",439,{"name":49,"slug":50,"count":51,"latest_published_at":52},"Deals","deals",317,"2026-10-01T22:00:00.000Z",{"name":54,"slug":55,"count":56,"latest_published_at":18},"Hardware","hardware",199,{"name":58,"slug":59,"count":60,"latest_published_at":18},"Science","science",176,{"name":62,"slug":63,"count":64,"latest_published_at":65},"Consumer Tech","consumer-tech",155,"2026-10-01T19:54:10.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Dev Tools","dev-tools",96,"2026-10-01T16:57:03.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",93,"2026-09-30T21:41:11.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Startups","startups",90,"2026-10-01T21:55:22.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Gaming","gaming",53,"2026-10-02T02:50:39.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",7,"2026-10-01T09:00:00.000Z"]