[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-new-benchmark-exposes-ai-agents-confusing-repetition-for-proof":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},6482,"new-benchmark-exposes-ai-agents-confusing-repetition-for-proof","New Benchmark Exposes AI Agents Confusing Repetition for Proof","A new test called GraphEcho shows language model agents often mistake retracing the same path for finding new evidence, and fixing that can hurt accuracy.","AI agents that explore knowledge graphs can confuse walking the same path twice with finding new proof.\n\nResearchers built a benchmark called GraphEcho (arXiv:2609.17695) to test whether large language model agents treat repeated encounters with the same evidence as independent corroboration when they follow multiple paths through a graph. The benchmark holds the underlying evidence content fixed while varying how many paths lead to it and where those paths originate, then checks both what the agent concludes and how it explores. Across every frozen agent tested, adding more redundant paths to the same evidence increased the share of repeated walks: the models kept retracing ground they had already covered. A provenance-aware post-training method, called PAPT, cut down on that repetition and boosted accuracy on the synthetic benchmark, but it also explored fewer distinct sources overall.\n\nThe tradeoff got worse on real scientific claims, where PAPT still reduced repeat visits but accuracy dropped. That is the paper's central worry: an agent can learn to look efficient, generating fewer redundant loops, while actually gathering less real evidence, which is exactly the kind of improvement a demo would highlight without catching the underlying problem.\n\nIt is a reminder that an AI agent's apparent confidence often measures how many times it walked the same hallway, not how many rooms it actually checked.","[\"ai agents\",\"benchmarks\",\"llm research\",\"ai safety\"]","2026-09-17T04:00:00.000Z","2026-09-17T20:17:26.975Z","2026-09-17T20:17:38.848Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Add the arXiv citation\u002Flink (e.g., arXiv:2609.17695) so the benchmark and researchers are traceable to a specific source rather than attributed only to unnamed 'researchers.'","resolved","ai",[32,33,34,35],"ai agents","benchmarks","llm research","ai safety",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.17695",0,{"sections":42},[43,47,51,56,61,65,69,74,79,83,88,93,98,103],{"name":44,"slug":30,"count":45,"latest_published_at":46},"AI",3852,"2026-09-17T08:27:09.000Z",{"name":48,"slug":49,"count":50,"latest_published_at":18},"Security","security",648,{"name":52,"slug":53,"count":54,"latest_published_at":55},"Policy","policy",338,"2026-09-11T04:00:00.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":18},"Hardware","hardware",154,{"name":66,"slug":67,"count":68,"latest_published_at":18},"Science","science",114,{"name":70,"slug":71,"count":72,"latest_published_at":73},"Consumer Tech","consumer-tech",99,"2026-09-09T17:27:33.000Z",{"name":75,"slug":76,"count":77,"latest_published_at":78},"Software","software",75,"2026-09-10T20:41:21.000Z",{"name":80,"slug":81,"count":82,"latest_published_at":18},"Dev Tools","dev-tools",73,{"name":84,"slug":85,"count":86,"latest_published_at":87},"Startups","startups",55,"2026-09-09T23:14:29.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"Gaming","gaming",43,"2026-09-10T12:18:06.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"General","general",41,"2026-09-08T01:57:23.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":104,"slug":105,"count":106,"latest_published_at":107},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]