Cutting an AI agent's decision costs does not reliably cut its failure rate, according to a new study.
Researchers tested five AI agents across two very different task types: physical household chores in the ALFWorld simulator and mobile app navigation in AndroidWorld. Swapping in Jev, a faster and cheaper decision-making approach, cut decision cost and latency for every one of the five agents across 4,500 total task runs. But task success held roughly steady only on the household tasks. On two of the mobile agents, success rates dropped more sharply. The researchers traced the shortfall to agents spending their savings on extra reasoning steps and interactions, then failing to recover from mistakes or declaring a task finished before it actually was.
That gap matters because efficiency numbers like cost-per-decision can look great while the metric users actually care about, successful task completion, quietly gets worse. In one agent, the cost per successful outcome rose even as average spending per attempt fell, meaning a cheaper decision engine ended up costing more per actual result.
It is a useful check on any vendor pitch built around 'faster inference, same quality.' Anyone evaluating agent tooling should ask for cost per completed task, not just cost per decision.