A new benchmark hands AI coding agents a half-built video-game world model and a compute budget, then checks whether they can improve it without anyone telling them how.
AutoWorldModel-Bench is a closed-loop test built around eight different game environments. Coding agents get a baseline world model (the kind of system that predicts what happens next in a game) and have to edit, retrain, and test it themselves under a fixed compute budget. The benchmark strips out raw pixels and instead feeds agents each game's actual state data in a shared format, so the agents are graded on modeling how the game evolves, not on recognizing what is on screen. Across 64 runs, Codex 5.4 and Claude Opus 4.6 raised their score on a held-out test in all but one session, with about half producing a substantial gain and the rest smaller but still positive. In 91% of those sessions, the winning move was a real change to the model or training setup, not a minor settings tweak.
That distinction matters more than the headline win rate. Most agent benchmarks today hand out a spec and grade agents on hitting it, closer to contract work than research. This one has no spec, just a scoreboard, which is a rougher approximation of what actual ML research looks like: try something, measure it, decide what to try next. The fact that agents mostly reached for structural changes rather than hyperparameter fiddling suggests they were doing something closer to reasoning about the problem than brute-forcing a config file.
Still, one benchmark suite with eight games is a narrow slice of research. Whether either model's editing instincts generalize to a world model nobody built as a benchmark toy is a separate, harder question this paper does not answer.