Toollery trims a bloated tool list down to a handful of real candidates before an AI agent has to choose.
Researchers built Toollery as a training-free system for LLM agents that need to pick from hundreds or tens of thousands of tools, skills, or API functions. Instead of stuffing every option into the prompt, it pre-generates likely user queries for each tool description, builds a search index from those queries, and uses it to narrow any real request down to a small set of candidates - the paper tests a top-10 budget - before the model makes its final call. The team ran it on three benchmarks: SkillRouter, a set of roughly 79,000 capabilities; BFCL-V4, which covers more than 440 atomic tools; and a private dataset of 3,396 requests from an in-car "smart cockpit" assistant spanning 220 tools. At that fixed top-10 budget, Toollery improved end-to-end selection accuracy on the cockpit data, held accuracy steady on BFCL-V4, and beat plain retrieval on recall across the board.
Every tool description an agent reads costs tokens, adds latency, and gives the model one more chance to pick wrong - a problem that only gets worse as agent platforms and app marketplaces keep adding integrations. Toollery's answer is a retrieval layer bolted on before the decision step, not a retrain of the model or the tools, which makes it the kind of change a team could actually ship next sprint rather than next quarter.
The authors are upfront that the payoff isn't free: results depend on how well the anticipated queries match real usage, and on whether a candidate list that changes every request still plays nicely with a provider's prompt caching. That caveat is worth remembering before anyone calls this "solved."