A guardrail built to catch AI jailbreak attempts can be reverse-engineered using the very techniques meant to explain its decisions.
Researchers ran an exploratory case study on Prompt Guard 2, a classifier widely used as a first line of defense against prompt injection and jailbreak attacks on large language models. Using explainable AI methods (Vanilla Gradient and SHAP attribution), they found the classifier's calls rest on the cumulative weight of many tokens, not a handful of obvious red flags. Armed with that map, they used saliency-guided synonym swaps and sentence-level paraphrasing to flip the classifier's verdict while changing only a moderate share of the text - in some cases producing a working jailbreak against the underlying LLM. A dataset-scale analysis found that injection prompts already slipping past the classifier systematically lack the lexical markers it relies on.
That's the uncomfortable part: the same transparency tools designed to help defenders audit a guardrail double as a blueprint for beating it. Any classifier-based filter that leans on legible lexical patterns - not just Prompt Guard 2 - is exposed to the same trick, since paraphrasing costs an attacker little and barely changes what a human reader would flag as suspicious.
Guardrails like this are typically pitched as a first line of defense, not the last one. This paper is a reminder that "first line of defense" and "static target" are not the same thing, and that explainability research, however well-intentioned, hands attackers a diagnostic tool too.