Researchers have worked out the exact mathematical condition that decides whether an AI jailbreak filter can be proven, not just tested, to catch malicious prompts.
Safety heads are small classifiers bolted onto language models to catch harmful prompts before the model answers them. A new paper asks a sharper question: can you mathematically guarantee a safety head will not flip its verdict if an attacker nudges the input just slightly? The answer hinges on one number: the l-infinity norm of the model's state transition matrix. Keep it below 1, the so-called contraction condition, and the system is provably stable, with certified accuracy on toxic comment data jumping from 41% to 59% once that constraint is enforced. Applied to jailbreak detection, a contraction-regularized S4 safety head trained on JailbreakBench transferred zero-shot to AdvBench with a 99.4% detection rate and to HarmBench at 98.8%.
Here is the part that undercuts the headline number: a plain logistic regression run on mean-pooled Mamba-130M embeddings matched or beat that fancy certified S4 head on every detection metric. Harmful intent, it turns out, is already linearly separable in embedding space, no exotic architecture required. The real contribution isn't better detection. It's a formal guarantee that today's red-team-and-hope safety filters don't offer.
The HarmBench number sounds airtight until you do the math: 98.8% detection means roughly one in eighty-three attack attempts still gets through, a reminder that provable robustness means consistency under small perturbations, not real-world infallibility.