Security/ prompt-injection · ai-security · ai-agents · llm-safety

Researchers Find LLMs Silently Flag Prompt Injection Attempts

AI agents' hidden states can already detect prompt-injection attacks, but new research shows the models rarely act on that knowledge without help.

AI agents can apparently sense when they're being tricked - they just don't reliably do anything about it.

A new study probed eight large language models, including the 753-billion-parameter GLM-5.2 and the 2.8-trillion-parameter Kimi-K3, to see whether their internal hidden states carry a signal for indirect prompt injection, the trick where a malicious instruction is buried inside a tool result, a webpage, or some other input the agent processes rather than typed directly by a user. Simple linear probes trained on those hidden states predicted exposure to injected instructions with 0.90+ AUROC, even on attacks, instructions, and task types the probes had never seen, and even when attackers adapted their approach or switched languages. That is a strong signal sitting largely unused inside models that keep getting fooled anyway.

The researchers call this a knowledge-action gap: the model's internal representations flag that something is off, but current training does not reliably translate that into refusing the malicious side-task. Their fix, a probe-gated reasoning defense applied at test time, cut the attack success rate on tough AgentDojo benchmarks from 34.6% to 0% on Qwen3.5-27B, while doing less damage to the model's ability to complete legitimate tasks than existing defenses.

It is a useful reframing of agent security: instead of only filtering inputs or hardening prompts, this treats the model's own hidden states as an underused sensor. Whether that sensor generalizes past benchmark attacks to the messier, real-world tool calls agents actually make is the open question.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →