[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-acne-grading-ai-looks-great-on-paper-but-flops-on-new-patients":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},9608,"acne-grading-ai-looks-great-on-paper-but-flops-on-new-patients","Acne Grading AI Looks Great on Paper but Flops on New Patients","The LENS-GRF model aced lab benchmarks but underperformed a simple baseline on a new patient cohort, exposing gaps in medical AI validation.","A new AI model for grading acne severity posted near-96% accuracy in the lab, then came in under a simple baseline on a fresh set of patients.\n\nResearchers built LENS-GRF, which pairs a whole-face image classifier with a lesion-by-lesion detector that locates and counts individual pimples and cysts, then combines the two readings into a four-class severity score. On the ACNE04 benchmark, the fully automated version reached 80.82% accuracy; fed hand-labeled lesion locations instead of its own detector's output, it climbed to 95.89%. The same audit also turned up 15 duplicate images split across ACNE04's training and test sets, five of them carrying conflicting severity labels. When the team ran a locked zero-shot test on a separate 200-patient, 600-image cohort, the fully automated model scored just 35% accuracy, below a bare-bones baseline's 42.5%.\n\nThat collapse outside the home benchmark is the real story: a medical AI's lab score, especially one built on a dataset with known duplicate and mislabeled images, says little about how it performs on patients it has never encountered. The paper's own dermatologist comparison underlines how hard this task is for anyone: with perfect lesion annotations handed to them, the best of three practicing dermatologists topped out at 47.97% accuracy, and the dermatologists agreed with each other only 49% to 66% of the time.\n\nIn other words, this isn't a confident expert versus a faltering machine. It's two unreliable narrators comparing notes.","[\"ai\",\"healthcare\",\"computer-vision\",\"dermatology\"]","2026-10-02T04:00:00.000Z","2026-10-03T02:46:04.714Z","2026-10-03T02:46:10.819Z","published",null,[],"ai",[24,26,27,28],"healthcare","computer-vision","dermatology",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.00294",0,{"sections":35},[36,39,43,47,52,56,60,65,70,75,80,85,90,95],{"name":37,"slug":24,"count":38,"latest_published_at":18},"AI",5896,{"name":40,"slug":41,"count":42,"latest_published_at":18},"Security","security",837,{"name":44,"slug":45,"count":46,"latest_published_at":18},"Policy","policy",438,{"name":48,"slug":49,"count":50,"latest_published_at":51},"Deals","deals",317,"2026-10-01T22:00:00.000Z",{"name":53,"slug":54,"count":55,"latest_published_at":18},"Hardware","hardware",199,{"name":57,"slug":58,"count":59,"latest_published_at":18},"Science","science",171,{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",155,"2026-10-01T19:54:10.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Dev Tools","dev-tools",96,"2026-10-01T16:57:03.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Software","software",93,"2026-09-30T21:41:11.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Startups","startups",90,"2026-10-01T21:55:22.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Gaming","gaming",53,"2026-10-02T02:50:39.000Z",{"name":86,"slug":87,"count":88,"latest_published_at":89},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":91,"slug":92,"count":93,"latest_published_at":94},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"How-To","how-to",7,"2026-10-01T09:00:00.000Z"]