AI agents waste GPU memory guessing how long a tool call will take, when the tool already knows the answer.
A new paper proposes that agentic tool calls report their own progress while running, instead of forcing the serving system to guess. Right now, systems estimate a tool's duration from its name, past history, or a duration declared in advance, then decide whether to keep, evict, or reload the request's KV cache from GPU memory. The researchers tested four public agent corpora and found a readable progress signal in most tool calls once it is surfaced. A harness recovers this signal without changing what the agent sees, and with no cost to benchmark performance.
This matters because KV cache eviction decisions happen constantly in production agent systems, and bad guesses mean either wasted memory or slow reloads. The paper reports the self-reported signal is several times to an order of magnitude more accurate than the best existing predictors, and unlike those predictors, it holds up when conditions change. Fed into a production engine, it cut p90 time-to-first-token after a tool call by roughly 20%, landing close to a theoretical best case.
The fix here is almost embarrassingly simple: stop predicting and start listening. It is a reminder that a lot of agent infrastructure is still solving problems by proxy — inferring state instead of asking for it directly — and that the plumbing is often the bottleneck, not the model.