Researchers built a caching system that watches which AI agent is about to run next, so servers stop redoing work they already finished.
Multi-agent LLM setups split a single request across several specialized agents, each rerunning the same system prompts, tool definitions, and examples. That repetition should make cached results reusable, but most serving systems evict those cached contexts using simple recency rules, so the cache is often gone by the time an agent runs again. A team describes CacheScout, a runtime layer built on top of vLLM that learns, in real time, the order in which agents actually run, without needing a predefined workflow map or offline training. It then uses that learned pattern to decide what to keep in cache and what to prefetch before an agent needs it, without changing anything on the serving critical path.
The gains are concrete: 10 to 18 percentage points higher cache hit rates, 18 to 45 percent faster time to first token, 29 to 38 percent lower per turn latency, and up to 57 percent higher peak throughput in the workloads tested. For anyone running agent pipelines at scale, that is the difference between a system that pays for redundant computation on every hop and one that does not.
It is a reminder that most agentic AI costs right now are infrastructure costs, not model costs, and the more interesting engineering is happening in the plumbing, not the prompts.