[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-mlcommons-benchmark-puts-jailbreak-risk-at-1865-percent":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},10189,"mlcommons-benchmark-puts-jailbreak-risk-at-1865-percent","MLCommons Benchmark Puts Jailbreak Risk at 18.65 Percent","MLCommons' new benchmark found unsafe AI responses rise from 11.08% to 18.65% under jailbreak attempts, a 7.57 point Resilience Gap, not a doubling.","A new benchmark puts a hard number on how often AI models fail under jailbreak pressure.\n\nMLCommons released version 1.0 of its Jailbreak Benchmark, a standardized method for testing how eight open-weight AI systems handle attempts to bypass their safety guardrails. Researchers ran 264 seed prompts across eleven hazard categories, pairing a baseline test with an adversarial one that used attacks drawn from the MLCommons Jailbreak Taxonomy. Every response was graded against the AILuminate Assessment Standard. Across all systems and attacks, the unsafe-response rate rose from 11.08% at baseline to 18.65% under jailbreak conditions, a 7.57 point gap the benchmark calls the Resilience Gap.\n\nThat gap is the real headline. It is not that models are wildly unsafe by default, it is that phrasing a request as an attack measurably increases the odds of an unsafe answer. The benchmark also found more accessible systems, the kind built to be easy to run or fine-tune, showed a wider Resilience Gap than their more locked-down counterparts, which tracks with the long-standing tradeoff between openness and guardrail durability.\n\nCall it a report card, not a verdict: a reproducible 7.57 point gap is a useful baseline for comparing models, but it also means every one of them still gets worse once someone tries.","[\"jailbreak\",\"ai-safety\",\"benchmarks\",\"mlcommons\"]","2026-10-05T04:00:00.000Z","2026-10-06T06:04:44.645Z","2026-10-06T06:04:50.040Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The headline and dek claim unsafe replies 'nearly doubled,' but 18.65% over 11.08% is a roughly 68% increase, not a near-doubling — rewrite the headline\u002Fdek to state the actual rate change (11.08% to 18.65%, a 7.57-point Resilience Gap) without the inflated doubling claim.","resolved","security",[32,33,34,35],"jailbreak","ai-safety","benchmarks","mlcommons",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.02827",0,{"sections":42},[43,48,52,57,62,67,72,77,82,87,92,97,102,107],{"name":44,"slug":45,"count":46,"latest_published_at":47},"AI","ai",6335,"2026-10-05T18:29:00.000Z",{"name":49,"slug":30,"count":50,"latest_published_at":51},"Security",875,"2026-10-05T17:21:11.000Z",{"name":53,"slug":54,"count":55,"latest_published_at":56},"Policy","policy",455,"2026-10-05T18:15:20.000Z",{"name":58,"slug":59,"count":60,"latest_published_at":61},"Deals","deals",353,"2026-10-05T18:35:08.000Z",{"name":63,"slug":64,"count":65,"latest_published_at":66},"Hardware","hardware",209,"2026-10-05T15:55:29.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":71},"Science","science",182,"2026-10-05T18:58:12.000Z",{"name":73,"slug":74,"count":75,"latest_published_at":76},"Consumer Tech","consumer-tech",161,"2026-10-05T18:18:56.000Z",{"name":78,"slug":79,"count":80,"latest_published_at":81},"Software","software",103,"2026-10-05T18:23:19.000Z",{"name":83,"slug":84,"count":85,"latest_published_at":86},"Dev Tools","dev-tools",99,"2026-10-05T10:47:06.000Z",{"name":88,"slug":89,"count":90,"latest_published_at":91},"Startups","startups",94,"2026-10-05T18:07:07.000Z",{"name":93,"slug":94,"count":95,"latest_published_at":96},"Gaming","gaming",54,"2026-10-05T19:07:04.000Z",{"name":98,"slug":99,"count":100,"latest_published_at":101},"General","general",52,"2026-10-05T17:01:27.000Z",{"name":103,"slug":104,"count":105,"latest_published_at":106},"Reviews","reviews",33,"2026-10-05T11:57:17.000Z",{"name":108,"slug":109,"count":110,"latest_published_at":111},"How-To","how-to",8,"2026-10-05T09:00:00.000Z"]