Most prompt injection filters don't just miss attacks sometimes. New research suggests they're often confidently wrong, and don't know it.
A paper posted to arXiv on August 28 introduces the Latent Diagnostic Taxonomy, a framework for building a classifier and then auditing its own decisions. The method tunes the classifier's embedding dimensionality through cross-validation instead of guessing a fixed size upfront, then isolates a small set of latent support vectors, about 29% of training examples, that reveal which single tokens can flip a prediction. Those tokens get sorted into a taxonomy: safe to trust, heuristic bias, heuristic override, or insufficient context requiring human review. Tested on a public prompt injection dataset, the researchers found that 77% of the classifier's confident decisions flipped when a single token was removed.
That 77% figure matters more than the framework itself. It means the industry's default move, bolt a classifier in front of an LLM and call it a safeguard, may be building false confidence rather than real security. The paper's split between calibration failures and genuinely exploitable shortcuts gives defenders a way to tell which brittle decisions are annoying versus which ones an attacker could actually weaponize.
It's a diagnostic tool, not a fix. Nothing here makes a classifier harder to fool. It just tells you, after the fact, which of its confident answers you should have doubted all along.