[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-editing-an-ais-facts-quietly-wrecks-its-ability-to-judge-evidence":10,"sections":45},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":34,"tags":35,"sources":40,"feedback":44,"feedback_at":22,"cost_usd":44,"total_tokens":44},7716,"editing-an-ais-facts-quietly-wrecks-its-ability-to-judge-evidence","Editing an AI's Facts Quietly Wrecks Its Ability to Judge Evidence","Knowledge editing can leave benchmark accuracy untouched while quietly destroying a model's ability to weigh new evidence against what it already knows.","A new study finds that patching a language model's memory can quietly wreck its judgment, even when every standard test says nothing changed.\n\nResearchers ran 1,000 sequential edits on Qwen2.5-7B-Instruct using a conservatively tuned LoRA method. MMLU accuracy held steady to four decimal places, the usual sign an edit is safe. But the model's ability to arbitrate between its remembered answer and a contradicting injected passage fell apart: the spread of that arbitration signal dropped 36%, and error on the model's most confident quarter of decisions rose from 0.217 to 0.342, a roughly 58% jump. A random perturbation severe enough to crash MMLU from 0.6275 to 0.3725 did less of this specific damage than the popular MEMIT editing method did while MMLU sat at a healthy 0.605. Under retrieval with a frozen retriever, accuracy fell from 0.592 to 0.46. In three of five model-and-method combinations tested, MMLU collapsed to chance at 1,000 edits, while edit success still read 1.00 and locality checks still looked clean.\n\nThis matters because the standard tests for knowledge editing only check three things: did the fact change, do paraphrases follow, and did unrelated answers stay put. None of those catch a model quietly losing the ability to tell a good source from a bad one. That's a real liability for any retrieval-augmented system patched this way, since it's exactly the skill RAG depends on.\n\nEdit success and locality scores can read perfectly clean on a model that's already lost the plot. Worth remembering before anyone ships \"surgical\" model patches as a substitute for retraining.","[\"knowledge editing\",\"llm evaluation\",\"retrieval augmented generation\",\"ai research\"]","2026-09-25T04:00:00.000Z","2026-09-25T06:07:27.198Z","2026-09-25T06:07:32.987Z","published",null,[24,30],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"publisher-r1","publisher",1,"The body contradicts itself on MMLU accuracy at 1,000 edits — first stating it 'stayed unchanged to four decimal places' under the edit regimen, then later stating 'three of five model-and-method combinations collapsed to chance-level MMLU at 1,000 edits.'","resolved",{"id":31,"reviewer":26,"round":32,"reason":33,"status":29},"publisher-r2",2,"The claim that error 'nearly doubled' from 0.217 to 0.342 is mathematically wrong — that's roughly a 58% increase, not a near-doubling (doubling would put it near 0.434).","ai",[36,37,38,39],"knowledge editing","llm evaluation","retrieval augmented generation","ai research",[41],{"name":42,"url":43},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.29587",0,{"sections":46},[47,50,55,60,65,70,75,80,85,90,95,100,104,109],{"name":48,"slug":34,"count":49,"latest_published_at":18},"AI",4466,{"name":51,"slug":52,"count":53,"latest_published_at":54},"Security","security",729,"2026-09-24T19:54:21.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Policy","policy",386,"2026-09-24T23:50:55.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Deals","deals",237,"2026-09-24T22:00:00.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Hardware","hardware",182,"2026-09-25T01:25:53.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",138,"2026-09-24T18:24:52.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Consumer Tech","consumer-tech",128,"2026-09-24T19:24:34.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Software","software",88,"2026-09-24T23:06:55.000Z",{"name":86,"slug":87,"count":88,"latest_published_at":89},"Dev Tools","dev-tools",79,"2026-09-22T22:21:13.000Z",{"name":91,"slug":92,"count":93,"latest_published_at":94},"Startups","startups",71,"2026-09-24T20:45:00.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"Gaming","gaming",46,"2026-09-24T17:52:29.000Z",{"name":101,"slug":102,"count":98,"latest_published_at":103},"General","general","2026-09-25T02:12:57.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"Reviews","reviews",30,"2026-09-24T20:07:31.000Z",{"name":110,"slug":111,"count":112,"latest_published_at":113},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]