AI agents can apparently sense when they're being tricked - they just don't reliably do anything about it.
A new study probed eight large language models, including the 753-billion-parameter GLM-5.2 and the 2.8-trillion-parameter Kimi-K3, to see whether their internal hidden states carry a signal for indirect prompt injection, the trick where a malicious instruction is buried inside a tool result, a webpage, or some other input the agent processes rather than typed directly by a user. Simple linear probes trained on those hidden states predicted exposure to injected instructions with 0.90+ AUROC, even on attacks, instructions, and task types the probes had never seen, and even when attackers adapted their approach or switched languages. That is a strong signal sitting largely unused inside models that keep getting fooled anyway.
The researchers call this a knowledge-action gap: the model's internal representations flag that something is off, but current training does not reliably translate that into refusing the malicious side-task. Their fix, a probe-gated reasoning defense applied at test time, cut the attack success rate on tough AgentDojo benchmarks from 34.6% to 0% on Qwen3.5-27B, while doing less damage to the model's ability to complete legitimate tasks than existing defenses.
It is a useful reframing of agent security: instead of only filtering inputs or hardening prompts, this treats the model's own hidden states as an underused sensor. Whether that sensor generalizes past benchmark attacks to the messier, real-world tool calls agents actually make is the open question.