AI agents that plug into tools via the model context protocol (MCP) are still bad at spotting attacks that don't sound like attacks.
What actually happened: Researchers tested whether standard alignment fine-tuning could teach large language models to refuse "falsely benign" attacks (FBAs), prompt injections phrased as ordinary requests, like asking an agent to add an SSH key to a bashrc file. Using direct preference optimization (DPO), the current state-of-the-art alignment method, they found refusal rates topped out at 47% across five popular open-source models, no matter how much fine-tuning was applied. To do better, the team built RAG-Pref, a retrieval-augmented technique that feeds a model examples of preferred and dispreferred responses at inference time instead of retraining it. Paired with existing offline alignment, RAG-Pref delivered an average 3.7-fold improvement in FBA refusals, compared with 2.9-fold for other online methods and 1.5-fold for offline alignment alone.
Why it matters: Most AI safety tuning was built for the "how do I build a bomb" era of red-teaming, not for attacks disguised as routine developer tasks. MCP is already an open standard used to wire agents into real tools and file systems, so a gap here is a gap in production, not a lab exercise. RAG-Pref also improved general chat quality on AlpacaEval 2 and MT-Bench, unlike other online alignment methods, which the researchers note tend to trade safety for helpfulness.
Even a 3.7-fold improvement is a mitigation, not a fix. It says more about how weak the baseline defenses are than how solved the problem is.