[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-thinking-out-loud-may-signal-uncertainty-better-than-answers":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},4579,"thinking-out-loud-may-signal-uncertainty-better-than-answers","Thinking Out Loud May Signal Uncertainty Better Than Answers","A new study finds that analyzing the reasoning chains of visual language models predicts errors more reliably than watching what the models actually say.","Researchers tested whether visual language models' internal reasoning steps could flag their own mistakes — and in most cases, they can.\n\nThe study ran four thinking-mode visual language models through identical adversarial image-question sets (POPE) and measured whether entropy — a proxy for uncertainty — in either the reasoning chain or the final answer could predict when a model was about to get something wrong. The results split cleanly across three behavioral types: Qwen3-VL-8B-Thinking showed near-total collapse in answer entropy (AUROC 0.492, essentially a coin flip), making its answers nearly unreadable as confidence signals. GLM-4.1V-9B-Thinking held up well (0.716). InternVL3-8B fell in between, generating reasoning chains on only half of samples and posting middling scores either way. Across all three thinking-mode models, chain entropy outperformed answer entropy on the subset where chains were generated — 0.647, 0.759, and 0.608 versus 0.492, 0.716, and 0.602 respectively. That advantage held strongly for Qwen and GLM, but the InternVL3 edge was marginal and statistically unreliable given only 17 false-positive samples. A 300-sample VQAv2 pilot backed the trend: chain entropy scored 0.680 versus 0.595 for answer entropy overall, with the gap widening to 0.733 versus 0.467 for free-form questions. A simple abstention gate built on chain signals lifted accuracy from 71.0% to 93.8% at 62.7% coverage, with no added inference cost.\n\nThe practical upshot matters because most production confidence measures today look only at the output token distribution — the equivalent of asking someone how confident they sound, not how confidently they reasoned. If chain entropy is the stronger signal in most conditions, systems that ignore the scratchpad are leaving diagnostic information on the table. The abstention result — a 23-point accuracy jump with no extra compute — is the kind of number that gets engineering teams to pay attention.\n\nThe caveat is that this is a small pilot and InternVL3's partial-thinking behavior muddies the picture; the researchers flag its chain-entropy advantage as unreliable with that sample size. Whether these patterns generalize beyond adversarial object-presence probes to the messier distribution of real-world queries remains an open question.","[\"ai\",\"machine-learning\",\"visual-language-models\",\"uncertainty\"]","2026-07-10T04:00:00.000Z","2026-07-10T05:33:25.153Z","2026-07-10T05:33:27.946Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The article states the VQAv2 chain entropy score as 0.733 vs 0.467 for free-form answers but omits the overall VQAv2 pilot figures (0.680 vs 0.595) without explanation, and more critically it drops the caveat from the source that InternVL3's chain-entropy advantage is 'marginal and statistically unreliable' (n_FP = 17) — presenting it alongside the other models as if the finding holds equally, which misrepresents the source material; revise to accurately characterize InternVL3's result and resto","resolved","ai",[30,32,33,34],"machine-learning","visual-language-models","uncertainty",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.08059",0,{"sections":41},[42,46,51,56,61,66,71,76,81,86,91,95,100,105],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Dev Tools","dev-tools",59,"2026-07-07T04:00:00.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":92,"slug":93,"count":89,"latest_published_at":94},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":101,"slug":102,"count":103,"latest_published_at":104},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":106,"slug":107,"count":108,"latest_published_at":109},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]