[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-probes-that-flag-ai-errors-were-often-just-detecting-noise":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},8911,"probes-that-flag-ai-errors-were-often-just-detecting-noise","Probes That Flag AI Errors Were Often Just Detecting Noise","A new audit shows a popular AI error-detection test was mostly picking up measurement artifacts, not real signal, until researchers fixed the setup.","A new arXiv paper argues that a popular way of predicting when a vision AI model is about to be wrong was mostly picking up an artifact of the test itself, not a real signal.\n\nVision transformers route information internally through expert gates, attention weights, and halting scores before producing an answer, and researchers have assumed these signals reveal something about correctness beyond the output alone. To test that, the authors scrambled the correctness labels while keeping the real routing data, so any signal should have been fake by design, yet a standard probe still flagged a 'routing advantage' in just over half of 600 confidence-only tests (308 of them, 51.3 percent). Tracing the cause, the team found the problem wasn't routing at all: it was selecting checkpoints by validation accuracy instead of validation loss. Once they fixed that selection method, the false positives disappeared completely, and the broader false-detection rate across 1,920 comparisons spanning every output type, which sat at 27.5 percent before the fix, fell to zero.\n\nThat matters beyond this one paper. It's a caution for interpretability research generally: a comparison method can look like it's revealing something real about a model's internals when it's actually an artifact of how checkpoints get picked, not of the model itself. The fixed version of the test wasn't just quieter. It was also properly sensitive, catching a deliberately implanted signal the broken version missed.\n\nNo press release will say this, but a chunk of the excitement around 'the model knows when it's wrong' research may be measurement noise wearing a lab coat.","[\"ai\",\"interpretability\",\"research-methods\",\"vision-transformers\"]","2026-10-01T04:00:00.000Z","2026-10-01T10:29:35.412Z","2026-10-01T10:29:41.510Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"publisher-r1","publisher",1,"The false-detection rate figures are internally inconsistent: the body first states the probe showed a false 'advantage' in just over half of 600 tests (>50%), then later says the false-detection rate was 27.5% before the fix, which contradicts the earlier figure.","resolved","ai",[30,32,33,34],"interpretability","research-methods","vision-transformers",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.38956",0,{"sections":41},[42,45,50,55,60,65,70,75,80,84,89,94,99,104],{"name":43,"slug":30,"count":44,"latest_published_at":18},"AI",5350,{"name":46,"slug":47,"count":48,"latest_published_at":49},"Security","security",801,"2026-09-30T22:18:23.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",429,"2026-10-01T02:26:17.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",298,"2026-09-30T21:00:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Hardware","hardware",196,"2026-09-30T13:00:00.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Science","science",157,"2026-09-30T15:00:56.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Consumer Tech","consumer-tech",149,"2026-09-30T22:57:11.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Dev Tools","dev-tools",93,"2026-10-01T02:30:48.000Z",{"name":81,"slug":82,"count":78,"latest_published_at":83},"Software","software","2026-09-30T21:41:11.000Z",{"name":85,"slug":86,"count":87,"latest_published_at":88},"Startups","startups",84,"2026-09-30T20:39:09.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"Gaming","gaming",51,"2026-09-30T16:24:30.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]