A new technique lets AI agents flag ambiguous instructions instead of guessing — and the gains are substantial.
Researchers introduced a prompt-based decomposition that separates two kinds of uncertainty: how confident an agent is in its next action, and how well-specified the incoming request actually is. When request uncertainty crosses a threshold, the agent asks for clarification rather than plowing ahead. To test it, the team created two new benchmarks — WebShop-Clarification and ALFWorld-Clarification — where half the tasks are deliberately underspecified. Across five large language models, the method improved clarification F1 on ALFWorld-Clarification by 73% over one baseline and 36% over another.
Most deployed agents fail silently on vague tasks, completing the wrong thing with full confidence. A system that knows what it does not know — and says so — is considerably more useful in practice, especially in consumer or enterprise settings where users routinely give underspecified commands. The fact that the method uses only prompt engineering matters: it works with black-box APIs, requires no labeled training data, and adds no meaningful latency.
The gains held across all five model backbones tested, which suggests this is not a trick tuned to one architecture — though the benchmarks are new and self-authored, so independent replication would be a reassuring next step.