A new benchmark checks whether an AI system can actually learn something new, not just recite what it already knows.
Researchers built ExplorationBench, a pair of sandboxes called AlienCode and AlienLogic that run on invented rule systems designed to contradict everyday knowledge. Because the rules are executable, every answer can be checked exactly, and because they clash with familiar facts, a model cannot bluff its way through with training-data recall. AlienCode packs 31 discovery targets into 70 tasks, and AlienLogic does the same with 24 targets across another 70 tasks. Each sandbox arrives with a manual that is deliberately flawed, feedback tied to specific tasks, and a fixed toolkit, forcing systems to explore first and get tested on held-out problems second. The researchers ran ten AI systems through the setup.
The headline finding is that the strongest systems really can pick up rules they have never seen and apply them correctly, which is a genuine test of exploration rather than pattern-matching. But the results are shaky: performance swings widely across runs, and letting a system explore longer sometimes erases progress it had already made instead of building on it.
That inconsistency is the real story here. It suggests today's reasoning systems are better at retrieving buried knowledge than at running a disciplined experiment loop - the same gap that separates a student who has memorized answers from one who can actually run a lab.