A new paper shows that the routing logic powering AI cost-saving systems can be manipulated to force expensive compute on every query.
Multimodal AI products increasingly use "cascade" architectures: a cheap model handles easy queries, and a pricier model takes over when the first one is not confident in its answer. Researchers found that this confidence signal can be spoofed. Their attack, called Forced Deferral, embeds a subtle visual trigger in input images that pushes the cheap model toward uncertainty, causing every query to escalate to the expensive model. The trigger is universal. One pattern generalizes across different inputs and model families, and it outperforms prior image-perturbation and prompt-injection baselines across datasets and deferral metrics.
The attack is notable for what it does not do: it never targets answer correctness. The outputs stay clean. What gets hit is the compute bill. For AI products running at scale, that creates an exposure that standard security thinking misses. An adversary can inflate infrastructure costs without ever surfacing in accuracy metrics or content filters.
Cascade architectures have become a practical necessity as inference costs remain high. That they now carry their own threat model is worth noting before more AI infrastructure bets on them.