[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-ai-helpers-know-when-theyre-unsure-but-rarely-say-so":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},8946,"ai-helpers-know-when-theyre-unsure-but-rarely-say-so","AI Helpers Know When They're Unsure but Rarely Say So","New research shows AI models rarely voice their own uncertainty, and when they do, poorly targeted hedges can mislead users more than silence.","A new study on human-AI collaboration finds that AI models usually have a decent internal sense of when they're about to get something wrong - they just rarely tell you.\n\nResearchers built a puzzle task where a human Helper tells an AI Worker where to place pieces, then tested three frontier vision-language models (GPT-4.1, GPT-5, and GPT-5.5). When asked to report a full probability distribution over candidate pieces, that signal was reasonably well-calibrated (an ECE, or calibration error, of 0.15, meaning the model's stated confidence closely tracked how often it was actually right) and useful for spotting mistakes (an AUROC of 0.65, a measure of how well a signal separates correct from incorrect placements, where 0.5 is a coin flip). The model's own token-level confidence, by contrast, was almost always near-certain (97% average) and poorly calibrated (ECE of 0.44). Despite having the better signal available internally, the models asked for clarification on only 3.5% to 16.7% of turns.\n\nThe stakes show up in a 210-person human study. Given only a model's default, unhedged message, people accepted 78% of wrong placements and could not tell good instructions from bad ones (an AUC of 0.50, meaning they were essentially guessing). Precise wording, and especially well-targeted hedges like \"I think this piece, but I'm not fully sure,\" cut wrong-move acceptance to 36% without scaring people off correct moves. But a hedge generated automatically from the model's own uncertainty score did not reliably help, since it inherited the weaknesses of that underlying signal - sometimes making people trust bad advice more, not less.\n\nThat is the uncomfortable finding for anyone building AI copilots: an AI that \"knows\" it is unsure is not the same as one that tells you usefully. A hedge is only as good as its calibration, and a poorly calibrated hedge is worse than silence.","[\"ai\",\"human-ai-collaboration\",\"vision-language-models\",\"research\"]","2026-10-01T04:00:00.000Z","2026-10-01T12:11:09.786Z","2026-10-01T12:11:14.182Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Define ECE, AUROC, and AUC in plain English on first use (e.g., calibration error and ability to distinguish right from wrong placements) before citing their values, since the draft currently cites all three cold.","resolved","ai",[30,32,33,34],"human-ai-collaboration","vision-language-models","research",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.39518",0,{"sections":41},[42,45,50,55,60,65,70,75,80,84,89,94,99,104],{"name":43,"slug":30,"count":44,"latest_published_at":18},"AI",5351,{"name":46,"slug":47,"count":48,"latest_published_at":49},"Security","security",801,"2026-09-30T22:18:23.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",429,"2026-10-01T02:26:17.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",298,"2026-09-30T21:00:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Hardware","hardware",196,"2026-09-30T13:00:00.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Science","science",157,"2026-09-30T15:00:56.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Consumer Tech","consumer-tech",149,"2026-09-30T22:57:11.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Dev Tools","dev-tools",93,"2026-10-01T02:30:48.000Z",{"name":81,"slug":82,"count":78,"latest_published_at":83},"Software","software","2026-09-30T21:41:11.000Z",{"name":85,"slug":86,"count":87,"latest_published_at":88},"Startups","startups",84,"2026-09-30T20:39:09.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"Gaming","gaming",51,"2026-09-30T16:24:30.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]