AI personal assistants love to promise they'll remember your life. A new benchmark suggests they can barely understand it.
Researchers introduced Life-Bench, a benchmark of more than 11,800 question-answer pairs across 10 tasks, built to test whether multimodal AI models can reason over someone's photo-based life history rather than just tag faces or objects in a single image. The tasks split into three tiers: identifying concepts, understanding specific events, and aggregating patterns across many photos. The dataset is synthetic but human-verified, and statistically modeled on real users' photo collections. The team also built LifeGraph, a knowledge-graph framework that lets models pull up relevant photos on demand instead of cramming everything into context at once.
The results are rough. Across four different retrieval methods tested on Life-Bench, accuracy dropped sharply as questions required more context, falling below 0.40 on the aggregated-reasoning tasks. No single method won across every category, which points to an architecture problem rather than a data problem.
Every major AI assistant pitch now promises it will remember your life. This benchmark is a reminder that remembering is the easy part. Understanding what it adds up to is still unsolved.