Current AI agents are bad at helping confused users figure out what they actually want.
Researchers published a paper arguing that the standard assumption baked into most AI agents — that users arrive with well-formed preferences — is wrong. When a user doesn't know enough about a domain to specify what they want, asking clarifying questions doesn't help. The team formalized this gap using a framework called CoPref, which models how users build preferences through conversation rather than simply revealing ones they already hold. They then built CoShop, an interactive benchmark where an agent must converse with and make recommendations for a user who is still learning what they want.
The results are a quiet indictment of how frontier models handle ambiguity. None of the five models tested broke 56% accuracy on CoShop after five turns of conversation — not because they couldn't find relevant items, but because the interactions did almost nothing to expand users' understanding of their own needs. The failure mode is less "agent can't search" and more "agent doesn't teach."
This reframes a debate that has mostly focused on retrieval quality and tool use. The harder problem may be pedagogical: can an agent recognize when a user needs education rather than clarification, and deliver it concisely enough to stay useful? Most current systems are not designed to do that, and this benchmark gives researchers a concrete way to measure the gap.