[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-ai-agents-pass-final-checks-but-still-flub-the-process":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},9560,"ai-agents-pass-final-checks-but-still-flub-the-process","AI Agents Pass Final Checks but Still Flub the Process","A new evaluation framework finds AI agents passing final checks still violate process rules over 90 percent of the time, an enterprise testing blind spot.","A new enterprise AI benchmark finds that agents can get the right answer and still break almost every rule along the way.\n\nResearchers built a continuous evaluation framework that checks not just an AI agent's final output but the steps it took to get there - tool selection, arguments, execution order, and database integrity. They tested it on two variants of a \"Business Value Determination\" skill inside an enterprise system called Value Aware Resiliency, running 240 trials across two agent harnesses and three models. Of 175 runs that passed every final numerical check, 162 - 92.6 percent - still tripped a process-level check the researchers' tools caught. A broader definition of \"passing\" did not help much either: 151 of 164 runs that cleared it still violated how the agent was supposed to get there.\n\nMost AI agent testing today grades the final answer, the way a teacher might grade only the number at the bottom of a math problem. This paper argues that is not enough for systems that touch live databases and APIs, since an agent can land on the right number while calling the wrong tools in the wrong order. The team's dependency-attribution step also cut a mean of 6.34 failed checks per run down to 2.65 actual root causes, which matters more for debugging than for marketing.\n\nIt is the agentic-AI equivalent of a unit test that only checks the return value and never looks at what got written to the database along the way.","[\"ai agents\",\"enterprise ai\",\"ai evaluation\",\"ai testing\"]","2026-10-02T04:00:00.000Z","2026-10-03T00:38:44.755Z","2026-10-03T00:38:50.745Z","published",null,[],"ai",[26,27,28,29],"ai agents","enterprise ai","ai evaluation","ai testing",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.01833",0,{"sections":36},[37,40,44,48,53,57,61,66,71,76,81,86,91,96],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",5896,{"name":41,"slug":42,"count":43,"latest_published_at":18},"Security","security",837,{"name":45,"slug":46,"count":47,"latest_published_at":18},"Policy","policy",438,{"name":49,"slug":50,"count":51,"latest_published_at":52},"Deals","deals",317,"2026-10-01T22:00:00.000Z",{"name":54,"slug":55,"count":56,"latest_published_at":18},"Hardware","hardware",199,{"name":58,"slug":59,"count":60,"latest_published_at":18},"Science","science",171,{"name":62,"slug":63,"count":64,"latest_published_at":65},"Consumer Tech","consumer-tech",155,"2026-10-01T19:54:10.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Dev Tools","dev-tools",96,"2026-10-01T16:57:03.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",93,"2026-09-30T21:41:11.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Startups","startups",90,"2026-10-01T21:55:22.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Gaming","gaming",53,"2026-10-02T02:50:39.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",7,"2026-10-01T09:00:00.000Z"]