A research framework called Incognita exposes how badly current AI agents mishandle tasks that require talking to people before doing anything.
Researchers built Incognita on top of Concordia, an existing simulation platform, to create what they call socially distributed task environments — settings where no single participant holds all the information needed to complete a task. Think of it like a retail support scenario where the customer, a billing specialist, and a shipping specialist each know different things. The agent must interview the right people before it can take any consequential action. Across 540 trials on 18 tasks, three generative agent models scored 0%, 8.9%, and 17.2% on final-state success — meaning even the best model failed more than four out of five times. Premature finalization, where an agent declares a task done before it actually is, dropped from 100% on the weakest model to 87% on the middle and 58% on the strongest.
The numbers matter because they reveal a specific failure mode that standard benchmarks tend to miss: agents act before they know enough. Most existing tests hand an agent a self-contained problem; Incognita withholds critical information behind role-isolated participants the agent must actively contact. Better models do more of that outreach — they elicit more hidden knowledge and attempt more grounded writes — but higher effort has not yet translated into reliable outcomes.
This is the social version of a known AI limitation: models that reason well in isolation still fall apart when they need to coordinate. Until agents can reliably decide when to ask and whom to ask before committing to an action, deploying them in any multi-stakeholder workflow is a gamble.