A new benchmark exposes a blind spot in how we evaluate AI agents: not whether they can solve problems, but whether they can manage other agents doing the solving.
Researchers introduced ClawArena-Team, a benchmark of 41 multi-turn, multimodal scenarios covering 258 evaluation rounds and 72 staged updates. The setup is deliberate: one lead model manages a fixed pool of subagents, handling delegation, parallel workloads, and dynamic workflows. The lead model sees only text and controls only part of the workspace, so score differences reflect management skill rather than raw capability. Scoring is execution-based — no LLM judge — using a metric called the Subagent-Management Score, which multiplies task correctness by a least-privilege and modality-routing factor. Twelve models were tested, spanning proprietary, community-hosted, and self-hosted options.
The findings are damning on one specific front: not a single model exceeded 50% precision on workspace-permission granting. That matters because over-privileged subagents are a real security and reliability risk in production pipelines. Equally striking is the cost-vs-quality gap — API costs varied by more than 100 times across models, while overall scores varied by less than 4 times, with cheap open models sitting on the Pareto frontier alongside expensive proprietary ones.
Most leaderboard scores clustered within a 9.9-point band, which sounds like near-parity until you note that underlying orchestration behaviors diverged by more than an order of magnitude — meaning similar scores can mask wildly different failure modes. The broader implication: as multi-agent deployments become standard infrastructure, the benchmarks used to select models are measuring the wrong thing.