A new benchmark called TANGLE tests whether AI agents can admit they don't have a clear answer when personal memories conflict.
Researchers built TANGLE, a set of 541 test cases spanning 40 personas, organized around three kinds of unresolvable memory conflict: preferences that depend on context, behavior that changes over time, and sources that contradict each other. Unlike prior benchmarks, which score an agent on picking the "right" memory out of conflicting evidence, TANGLE checks whether the agent recognizes that no single answer exists. The team evaluated models on both curated memory and memory extracted from realistic multi-session chats, scoring five things: whether the agent spots the conflict, reasons about its cause, calibrates confidence, asks for clarification, and stays faithful to the original memories. They also propose a fix called Conflict-Aware Action Policy, which adjusts an agent's response to the specific type of conflict instead of applying one fixed rule.
Personal-memory features are already shipping in consumer AI assistants, and this research suggests those assistants are bad at knowing when they don't know. Models were reasonably good at detecting that a conflict existed when given clean, curated memory, but far worse at turning that awareness into a hedge or a follow-up question. When memory had to be extracted automatically from conversation history, that recognition mostly broke down entirely.
A chatbot that confidently states your own preference wrong is a worse failure than one that just asks.