Jailbreak attacks on large language models work by disabling only part of the brain, not all of it.
Researchers studying the internal mechanics of jailbreak attacks found that successful exploits do not wipe out a model's safety features wholesale. Instead, they selectively suppress a specific class of attention heads — called Adversarially Compromised Heads (ACHs) — concentrated in the model's early layers. A separate class, Safety-Aligned Heads (SAHs), sits in the mid-layers and keeps firing even when an attack succeeds. Ablation experiments backed up the causal story: suppressing a small number of ACHs was enough to make a model comply with normally refused requests, while removing SAHs significantly weakened mid-layer safety signals.
This distinction matters because it reframes what a jailbreak actually is. It is not a full override of alignment — it is a targeted exploit of a structural weak point, while deeper safety representations persist. That persistence is exploitable in the defenders' favor: the researchers showed that simply reading those surviving activations, without any additional model training, produced competitive detection performance with strong resistance to adversarial inputs.
The practical upside is real but narrow. A training-free detector built on persistent safety signals is cheaper to deploy than a retrained model — but it still assumes attackers do not eventually learn to suppress SAHs too. The arms race, as usual, is not over.