[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-ai-agents-still-struggle-to-work-a-room":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},3655,"ai-agents-still-struggle-to-work-a-room","AI Agents Still Struggle to Work a Room","A new benchmark tests whether AI agents can gather info from multiple specialists before acting — and the results are not encouraging.","A research framework called Incognita exposes how badly current AI agents mishandle tasks that require talking to people before doing anything.\n\nResearchers built Incognita on top of Concordia, an existing simulation platform, to create what they call socially distributed task environments — settings where no single participant holds all the information needed to complete a task. Think of it like a retail support scenario where the customer, a billing specialist, and a shipping specialist each know different things. The agent must interview the right people before it can take any consequential action. Across 540 trials on 18 tasks, three generative agent models scored 0%, 8.9%, and 17.2% on final-state success — meaning even the best model failed more than four out of five times. Premature finalization, where an agent declares a task done before it actually is, dropped from 100% on the weakest model to 87% on the middle and 58% on the strongest.\n\nThe numbers matter because they reveal a specific failure mode that standard benchmarks tend to miss: agents act before they know enough. Most existing tests hand an agent a self-contained problem; Incognita withholds critical information behind role-isolated participants the agent must actively contact. Better models do more of that outreach — they elicit more hidden knowledge and attempt more grounded writes — but higher effort has not yet translated into reliable outcomes.\n\nThis is the social version of a known AI limitation: models that reason well in isolation still fall apart when they need to coordinate. Until agents can reliably decide *when* to ask and *whom* to ask before committing to an action, deploying them in any multi-stakeholder workflow is a gamble.","[\"ai\",\"benchmarks\",\"agents\",\"research\"]","2026-07-07T04:00:00.000Z","2026-07-07T05:02:47.753Z","2026-07-07T05:02:50.580Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The article states the worst-performing model succeeded on 'zero percent' of tasks, but the source shows the three models scored 0%, 8.9%, and 17.2% — the body omits the middle model entirely and misrepresents the dataset as only two models; additionally, the premature finalization figure for the middle model (87%) is dropped, leaving an incomplete picture that distorts the source findings.","resolved","ai",[30,32,33,34],"benchmarks","agents","research",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.02975",0,{"sections":41},[42,46,51,56,61,66,71,76,81,85,90,94,99,104],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":86,"slug":87,"count":88,"latest_published_at":89},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":91,"slug":92,"count":88,"latest_published_at":93},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]