A new benchmark called SimLife-BP tests something most AI evaluations skip entirely: can a model actually learn your habits over time, or is it just guessing based on what happened most recently?
Researchers built SimLife, a simulation platform that generates long-term household life data, including visual observations, action logs, and synthetic dialogue with audio. From that, they built SimLife-BP, a benchmark of 106 episodes averaging over 15 hours and nearly 39 in-game days each, totaling 1,439 question-answer pairs. The questions probe whether models can infer the actual rules behind a household's routines, not just predict the next likely action, testing direct, counterfactual, noisy, and inverse reasoning. The results were not flattering. Frontier models mostly leaned on frequency-based shortcuts, guessing based on what happened most often, rather than reasoning through if-then logic, and they struggled badly when routines changed partway through.
That gap matters more than it sounds. Every pitch for a household robot, a personal assistant, or a long-running agentic tool assumes the system will get better at understanding you the longer it watches. This benchmark suggests that assumption is shakier than the marketing implies: models can mimic pattern recognition over short windows but lose the thread once a routine shifts or noise enters the picture.
It is also a useful corrective to the current AI narrative. Most benchmarks reward short-context tasks where memorizing surface patterns looks like understanding. SimLife-BP is built to catch that difference, and frontier models mostly failed to clear it. Worth remembering next time a company demos an agent that promises to "learn your routine."