AI agents that call external tools are quietly failing millions of non-English speakers.
Researchers introduced MLCL, a diagnostic benchmark testing large language models on tool calling across Chinese, Hindi, and the low-resource language Igbo. The results complicate the rosy picture painted by English-only evaluations. The dominant failure mode is not misunderstanding the user's request — models generally get that right. The problem is that models output parameter values in the user's language rather than in the language-neutral format that underlying APIs expect, breaking execution silently.
Most LLM benchmarks are English-centric, so inflated scores hide real-world brittleness the moment a product goes global. Tool calling is not a cosmetic feature — it is the mechanism by which agents book flights, query databases, and execute code. A parameter mismatch at that layer means the action simply does not happen.
The researchers tested several inference-time fixes and found that each one meaningfully cut language-induced errors but none fully closed the gap with English-level performance. That last mile may require training-time solutions, not prompt engineering — which is a more expensive problem for the labs to solve.