A new benchmark suggests AI personal assistants often can't tell the difference between remembering something and making it up.
Researchers built MIRAGE (Multimodal Interaction Retrieval, Attribution, and Grounding Evaluation) to test how well multimodal AI agents use evidence from earlier in a conversation, including files, chat history, and workspace state, while holding the underlying facts and questions constant. The only thing they varied was conversation state: how deep into a chat the question landed, and whether it came before or after the conversation got compressed into a summary. Seven frontier and open-weight models were checked on three things: could they tell if a question was even answerable from prior evidence, could they find the right source, and could they answer from it. The failures did not follow one steady decline. The team found two separate, unpredictable failure patterns, one tied to how deep a conversation went before compression and another tied to what happened after.
That distinction matters because most agent benchmarks only grade the final answer, which lets a good guess look identical to genuine recall. Open-weight models were especially prone to this: they leaned on raw context and rarely switched to re-retrieving evidence through a tool once that context's origin got shaky. For products sold on the promise of a personal assistant that remembers your files and past conversations, that is a meaningful gap between marketing and mechanism.
An assistant that sounds sure and an assistant that actually checked its notes produce the same sentence, until someone builds a test that can tell them apart.