AI/ ai-agents · llm-research · human-ai-interaction · arxiv

New Framework Teaches AI Agents When to Ask Clarifying Questions

A new co-training method helps AI agents learn exactly when asking a clarifying question beats guessing, boosting accuracy on underspecified tasks.

A new training technique teaches AI agents to ask for clarification only when it will actually change the outcome.

Researchers built CLEAT (CLarification through valuE-guided Agent co-Training), a framework that trains two things at once: a controller that predicts whether a question is worth asking, and the task policy that decides what to do with the answer. The system estimates what the paper calls residual query value, the extra payoff from asking versus just proceeding, by comparing paired rollouts drawn from the same interaction history. Using the open-source model Qwen3-4B, the team tested the approach across four benchmarks and ten evaluation settings, and it scored highest in every one. The biggest gains were a 13.2 percentage point improvement over the strongest baseline on a travel-planning benchmark called UserGym Travel, and a 6.5 point gain on IN3, a test of judging when a request is underspecified.

Most agents that "ask a clarifying question" run on a hardcoded rule: ask when confidence is low, or ask whenever something looks ambiguous. CLEAT instead learns from outcomes when a question is worth the interruption, and keeps updating that judgment as the agent's own policy improves. The ablations even show the task policy gets better at handling incomplete information on its own, with the controller removed, which suggests the training process itself, not just the extra trigger, is doing real work.

It is a clean result on four benchmarks with a 4-billion-parameter model. Whether that same value judgment holds up on messier, higher-stakes assistants than a fictional travel booking agent is the obvious next question.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →