Researchers have found that the technique major cloud providers use to speed up LLM queries is structurally vulnerable to a class of integrity attacks.
Semantic caching works by storing LLM responses indexed to embedding vectors, so a query that means roughly the same thing as a previous one retrieves a cached answer instead of hitting the model again. AWS and Microsoft both use it. The new paper formalizes why that "roughly the same thing" property is the problem: for a cache to get high hit rates, nearby queries must map to nearby keys — but that locality is exactly what cryptographic hash functions are designed to prevent. The researchers built CacheAttack, an automated black-box framework that exploits this gap, achieving an 86% success rate at hijacking LLM responses and demonstrating it can push LLM agents into malicious behaviors without access to the underlying embedding model.
Prior work on semantic caching risks focused on side-channel leaks and privacy — what a cache reveals about other users' queries. This paper shifts the threat model to integrity: an attacker who can craft inputs that collide with cached keys can substitute arbitrary responses for legitimate ones, which is a qualitatively different and more dangerous problem in agentic contexts. A case study on a financial agent illustrates how that plays out when the model is taking actions, not just answering questions.
The trade-off the paper describes — performance versus collision resistance — is not a fixable bug but a structural property of how these caches are designed, which makes the mitigation conversation considerably harder than a patch.