AI/ ai · ai-agents · benchmarks · arc-agi

AI Agent Builds a Working Rulebook Before It Ever Wins

Twin, a test-time AI system, infers an unknown game's rules and goal from scratch, then plays ARC-AGI-3 levels more efficiently than most human testers.

A new AI agent skips the rulebook and writes its own, then beats the game anyway.

Researchers built Twin, a test-time world-model inference system in which a frontier coding agent writes an executable simulation of an unfamiliar game's rules and goal, using nothing but observation and interaction, not a hand-coded model built for that specific task. It checks its own work in a sandboxed twin of the game: the harness blocks any action until the program can reproduce every past observation, and each mismatch becomes a counterexample that patches the model. Tested on ARC-AGI-3, a benchmark of grid-based games with hidden rules, Twin cleared 179 of 183 levels (97.8%). On the 179 levels it cleared, it played more efficiently than human first-timers on 158 of them (88.3%), and it figured out the goal before ever seeing a reward signal on 156 of those levels (87.2%); the rest it solved by searching for the goal directly.

ARC-AGI-3 scores agents from 0 to 100 on a mix of completion and efficiency, benchmarked against humans playing each game for the first time. The base model alone scores 7.8% on that scale. Bolt on a generic off-the-shelf harness and it climbs to 61.1%. Give the same base model Twin's self-written world model and the score reaches 93.3%, with the system clearing 23 of 25 games in that run. The jump from 61% to 93% is the real story: it is not more raw model capability doing the work, it is the discipline of forcing an agent to write down and test a falsifiable theory of the game before it acts.

The paper's own admission is the honest part: building a usable world model turned out to be the easy half. Guessing what the game actually wants you to do is still the hard one.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →