Security/ prompt injection · ai security · llm agents · ai research

Prompt Injection Still Shifts Schema-Based AI Decisions

A study of Jev, a non-generative decision model, finds prompt injection rarely hijacks choices, though adaptive attacks nearly double success rates.

Researchers testing prompt injection attacks got a mixed result: hard to fool, but not immune.

The system in question is called Jev, a decision model that does not generate free-form text. Instead of writing a response, it picks one option from a fixed list and attaches a probability to each choice, a setup the researchers call a schema-defined or typed probabilistic output. The team ran 510 reconstructed test cases from InjecAgent, a benchmark that hides malicious instructions inside the content an AI agent processes, and found that this content shifted Jev's probability scores toward the attacker's preferred action but rarely flipped the final choice. Adding explicit override markers, essentially telling Jev to ignore embedded instructions, cut that influence further, while attacks that merely claimed relevance to the task had little effect.

The resistance was not absolute. When the researchers used score feedback to iteratively refine their attacks, the average highest probability assigned to an attacker's target action roughly doubled during optimization, and the success rate on fresh, unseen cases rose from 1.8 percent to 3.5 percent, with failures concentrated in cases where Jev's own initial preference was already narrow or the attacker controlled more of what Jev was reading.

Most prompt injection research targets generative agents that write open-ended text, where one hijacked output can cascade into a full derailment; this paper's real contribution is showing that swapping in typed, schema-constrained decisions shrinks the attack surface without closing it.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →