A new study says AI safety judges can be talked out of flagging harmful text without a single word of that text changing.
Researchers tested eight safety judges - including the deployed Llama Guard 4 and a GPT-4o-based grader - on over 600 replies from the JailbreakBench dataset, wrapping each harmful response in up to seven style tricks: a fake educational disclaimer, a bogus "reasoning" block, or a token refusal tacked onto an otherwise unchanged harmful body. The underlying content never moved, so any judge that changed its verdict was, by definition, wrong. Results were uneven: GPT-4o-mini flipped 19.9% of its correct "unsafe" calls when a token refusal was glued to the front, while Claude budged only 0.4% under the same trick. Llama Guard 4 flipped 12.3% of its harmful verdicts to "safe" just by framing the content as an "educational course."
This matters because these judges are the scoreboard for the entire jailbreak research field - every reported attack success rate and safety leaderboard runs through one of them. If a judge can be gamed by tone alone, some published "jailbreak" results may just be measuring which model is easiest to sweet-talk with formatting, not which model is actually less safe. The paper's cleanest evidence: swapping only the grading prompt on the same judge model cut the exploit tenfold, meaning the flaw sits in the grader's wiring, not the content it grades.
Two human annotators confirmed 90% of the flips were outright errors, and a second deployed guard, gpt-oss-safeguard-20b, resisted the trick entirely - proof this isn't an unsolvable problem, just an unevenly solved one.