A new benchmark says today's embodied AI agents barely learn from their own mistakes.
Researchers built EmbodiedCLBench, the first benchmark designed to test continual learning in embodied simulation, the kind of learning where a robot or virtual agent is supposed to carry skills from one task into the next. The setup has agents first tackle basic tasks, then face harder, unseen tasks that require recombining those basic skills. The team ran the benchmark across multiple agent harnesses and learning methods. Zero-shot agents, meaning no prior task experience, mostly failed the advanced tasks outright.
Giving an agent prior experience did help, and that benefit held steady even as task complexity increased, but the gains flattened fast: more learning samples did not translate into proportionally better performance, suggesting current methods cannot extract much from extra experience. That is a real problem for the growing hype around self-evolving agents that are supposed to get smarter the more they act in the world. This benchmark suggests that promise is outrunning what the field can currently deliver.
Think of it as the embodied AI equivalent of a student who crams the same lesson five times and still scores the same on the final.