A new study finds that the easiest way to beat an AI model's safety filters isn't a cleverer prompt. It's trying enough times.
Researchers tested best-of-N jailbreaking, which resamples a request with different framings until one slips past a defense, against SAGE, described as the strongest published self-check defense for open-weight models. When they paired repeated sampling with a code-completion encoding rather than surface-level noise like scrambled text, success rates jumped to 67%, 22%, and 15% of tested behaviors across three models, compared with a maximum of 4.7% for the encoding alone and 3.0% for character-based search at full budget. A follow-up test that isolated encoding from repetition showed the two failure modes are different: one defense collapsed under a deeper encoding, going from 7 to 67 successful behaviors, while another collapsed under broader resampling, going from 13 to 57. Even more telling, SAGE blocked 99.8% of individual attack attempts on one model, yet still lost 12 behaviors once attackers were allowed to keep trying, exposing a gap between per-attempt averages and real-world attacker persistence.
That gap matters because most safety benchmarks report defenses the way SAGE was reported here, as a mean success rate per attempt. An attacker doesn't average; they take the best of however many tries they get. The paper's fix isn't a patch or a new filter. It's architectural: stop building systems where the same generation step both answers the request and decides whether to answer it, because every retry redraws the safety verdict along with the response.
It's a familiar lesson dressed up in new numbers: any defense measured by average performance will look sturdier on a slide than it behaves against someone with a query budget and patience.