AI assistants that remember your life story still cannot give you good advice based on it.
Researchers introduced PRAGMA, a benchmark that tests whether AI memory systems can turn scattered past conversations into personalized recommendations, planning help, and decisions, not just retrieved facts. Unlike earlier memory benchmarks that mostly check whether a model can recall something you mentioned weeks ago, PRAGMA is built around longitudinal conversation histories with evidence annotations, shifting user preferences, and scenarios where the user's own assumptions have changed or turned out wrong. The team ran retrieval systems, dedicated memory architectures, and long-context models against these scenarios. Across the board, the systems struggled both to locate the right prior conversation and to reason with it once they found it.
Most pitches for personalized AI assume that a memory layer alone turns a chatbot into an assistant who truly knows you. PRAGMA's results suggest that gap is bigger than the marketing implies: pulling up a fact is a different skill from weighing months of changing context to make a real recommendation, and today's systems mostly fail at the second part.
Every company selling "memory" as a headline feature might want to sit with that distinction before the next product announcement.