An AI agent is only as good as the tools it can actually call - and a new framework tries to make sure it is calling the ones that work.
Researchers describe RATED (Reliability-Aware Tool Retrieval via Execution Dynamics), a system that picks which external tools an AI agent should use. It pairs a frozen dual-encoder, fine-tuned for semantic matching, with a lightweight contextual bandit that re-ranks tool candidates using real-time execution feedback - meaning it learns which tools are currently failing and adjusts its picks without retraining the whole model. The team also built EcoAgent, a benchmark of 5,172 real-world tools and more than 210,000 query-tool pairs, to test retrieval accuracy and reliability together. On that benchmark, RATED posted a 52.2% relative improvement in retrieval accuracy (MRR) over the ToolLLM API retriever, and cut the rate of picking a broken tool from 86.2% to 2.9% during a simulated outage with a 30% failure rate - all while adding less than 1 millisecond of latency.
Most tool-retrieval research treats availability as someone else's problem: find the semantically closest match and hope the API answers. That is a bad bet as agents increasingly chain together dozens of third-party tools with no uptime guarantees. Treating live failure data as a ranking signal, not an afterthought, is the more useful framing here, and the near-zero latency cost means it is not just a theoretical fix.
Still, this is one team's benchmark, built and graded by the same people who built the retriever, so the 86-to-3 percent headline number deserves a skeptical read until someone runs it on a different fleet of tools.