Turns out the sticker price on an AI model call is not what you actually pay.
A new paper on arXiv introduces the idea of "token inflation" - the gap between a model's advertised per-token cost and what an agentic system actually burns once retries are factored in. The researchers measured this gap across model tiers and task types, finding it as high as 4.25x for a 7B model working through multi-hop question answering. They built a system called InflationAgent that predicts difficulty before execution using a signal derived from local inference, then routes each query to the model tier that maximizes expected accuracy per real dollar spent, not per advertised token. On the GSM8K math benchmark, under a fixed budget, it hit 94.7% accuracy versus 91.0% for the existing FrugalGPT router, while using 31% fewer tokens.
This matters because most cost-routing systems, including the widely cited FrugalGPT, still price decisions off single-call cost. That is like budgeting a road trip by the price of gas per gallon and ignoring how many times you might get lost. The paper's most useful finding may be the failure mode it identifies: forwarding a failed reasoning chain to a stronger model, rather than starting fresh, cut GPT-4o's accuracy by up to 34.8 percentage points. Sloppy escalation does not just waste money. It actively makes the expensive model worse.
As agentic workflows spread, the real budget conversation is not about per-token pricing at all. It is about how often a system has to try again, and whether anyone is bothering to measure that.