[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-safety-rules-vanish-when-ai-agents-run-out-of-context":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},2841,"safety-rules-vanish-when-ai-agents-run-out-of-context","Safety Rules Vanish When AI Agents Run Out of Context","A new benchmark shows LLM agents that obey constraints perfectly can break them at a 59% rate once context compaction quietly drops the rules.","When an AI agent's memory gets trimmed, its rules can disappear with it.\n\nResearchers introduced a failure mode they call Governance Decay: long-running LLM agents rely on context compaction to stay within token limits, but that compression step can silently delete the safety constraints the agent was following. To measure this, the team built ConstraintRot, a benchmark of long-horizon scenarios with deterministic tool-call grading, and ran it across seven model families over 1,323 episodes. Violation rates rose from 0% when the policy was in full context to 30% after compaction, and hit 59% for the worst-performing models. When a constraint survived the summary, violations stayed at zero; when it was dropped, violations reached 38%.\n\nThe finding reframes where AI governance actually breaks down. Most safety research focuses on the model's weights or its system prompt — neither of which help once a running session compresses its own context and discards the rules mid-task. It also surfaces an attack vector: the researchers demonstrated a Compaction-Eviction Attack in which adversarial content in the context biases the summarizer to omit a legitimate policy, and optimized injections defeated every model tested.\n\nThe proposed fix — Constraint Pinning, which quarantines governance rules from lossy compression — restored violations to 0% on the benchmark without any model retraining. That sounds tidy, but benchmark-zero is not prod-zero, and the attack results suggest the underlying summarization step is itself a manipulable surface that a simple quarantine may not fully close.","[\"ai\",\"security\",\"llm\",\"ai-safety\"]","2026-06-30T04:00:00.000Z","2026-06-30T13:48:32.416Z","2026-06-30T13:48:35.337Z","published",null,[],"ai",[24,26,27,28],"security","llm","ai-safety",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.22528",0,{"sections":35},[36,40,44,49,54,59,64,69,74,79,84,88,93,98],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":26,"count":42,"latest_published_at":43},"Security",294,"2026-07-15T19:59:48.000Z",{"name":45,"slug":46,"count":47,"latest_published_at":48},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":50,"slug":51,"count":52,"latest_published_at":53},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":55,"slug":56,"count":57,"latest_published_at":58},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":60,"slug":61,"count":62,"latest_published_at":63},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":65,"slug":66,"count":67,"latest_published_at":68},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":70,"slug":71,"count":72,"latest_published_at":73},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":75,"slug":76,"count":77,"latest_published_at":78},"Dev Tools","dev-tools",59,"2026-07-07T04:00:00.000Z",{"name":80,"slug":81,"count":82,"latest_published_at":83},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":85,"slug":86,"count":82,"latest_published_at":87},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]