[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-ai-agents-handle-typos-fine-but-choke-on-flaky-tools":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},11336,"ai-agents-handle-typos-fine-but-choke-on-flaky-tools","AI Agents Handle Typos Fine but Choke on Flaky Tools","A new benchmark finds tool-using AI agents shrug off typos but lose up to 40% accuracy when APIs misbehave, and a training fix narrows the gap.","Researchers just built a benchmark that breaks AI tool-use agents the exact way real deployments already do, and it turns out most models are far shakier than their demo reels suggest.\n\nThe benchmark, called RobustBench-TC, throws 22 kinds of trouble at agents: typos and garbled inputs, duplicate or misconfigured tool names, bad reward signals and metadata, and broken runtime behavior like stalled timeouts. Every scenario is tied to a real, documented GitHub issue, not a hypothetical edge case. Across 21 models ranging from 1.5 billion to 32 billion parameters, including the closed-source o4-mini, typos barely mattered, costing under 5% accuracy. Bad reward signals and metadata cut accuracy by roughly 40%, and broken runtime behavior cut it by roughly 30% - and bigger models did not fix this on their own.\n\nThe researchers then trained a 3-billion-parameter agent on a mix of these failure types using a reinforcement-learning recipe called ToolRL-DR. That smaller model kept about three-quarters of its clean-conditions accuracy and matched open-source 14-billion-parameter baselines once things got messy, closing much of the gap to o4-mini. It even improved on runtime failures it was never trained on, which suggests the training builds a general habit of retrying and recovering rather than memorizing specific fixes.\n\nThat's the real finding here: robustness looks more like a training problem than a scale problem. The failures that actually take down tool-use agents in production are not the typos and odd phrasing everyone worries about - they are timeouts, duplicate tool names, and bad metadata, the boring infrastructure stuff.\n\nSo the next time an agent demo goes sideways, the polite assumption is user error. The less polite one, backed by this paper, is that someone's API timed out.","[\"ai-agents\",\"benchmarks\",\"reinforcement-learning\",\"tool-use\"]","2026-10-12T04:00:00.000Z","2026-10-12T08:53:10.419Z","2026-10-12T08:53:16.084Z","published",null,[],"ai",[26,27,28,29],"ai-agents","benchmarks","reinforcement-learning","tool-use",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.11928",0,{"sections":36},[37,40,45,50,55,60,65,70,75,80,85,90,95,100],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",6966,{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",952,"2026-10-12T04:54:12.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Policy","policy",496,"2026-10-11T20:06:43.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Deals","deals",489,"2026-10-12T13:30:00.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",237,"2026-10-12T13:24:45.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Science","science",196,"2026-10-11T15:34:40.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Consumer Tech","consumer-tech",192,"2026-10-12T14:07:13.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Startups","startups",121,"2026-10-12T13:22:34.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Software","software",115,"2026-10-11T07:34:45.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Dev Tools","dev-tools",111,"2026-10-12T11:22:24.000Z",{"name":86,"slug":87,"count":88,"latest_published_at":89},"General","general",70,"2026-10-11T14:15:00.000Z",{"name":91,"slug":92,"count":93,"latest_published_at":94},"Gaming","gaming",59,"2026-10-09T11:43:43.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"Reviews","reviews",35,"2026-10-12T14:00:00.000Z",{"name":101,"slug":102,"count":103,"latest_published_at":104},"How-To","how-to",8,"2026-10-05T09:00:00.000Z"]