Standard world-model checks leave a blind spot — and a new diagnostic exposes it.
Researchers introduced a diagnostic called operator-on-F that tests how well a learned world model's internal "latent" rollouts match the real environment's dynamics, rather than just asking whether the model predicts rewards accurately. The team tested it on TD-MPC2, a well-known model-based reinforcement learning architecture, across five model sizes on the cheetah-run benchmark. Reward-prediction error varied only from 0.028 to 0.091 across all sizes — roughly a 3x range — giving evaluators almost no signal to distinguish good models from bad ones. The two conventional metrics fared poorly: Bellman residual had a weak Spearman correlation of -0.10 with return, and reward error did only marginally better at -0.30.
Operator-on-F told a different story. Operator error ranged from 0.28 to 2.62 across the same sweep — nearly a 10x spread — and its rank correlation with return loss hit -0.90, with a bootstrapped confidence interval floor of -0.70. At the largest size tested, 317M parameters, operator error spiked to 2.62 while planning return collapsed to 0.9, even though reward-prediction error at that size (0.091) was unremarkable within its own narrow band. The diagnostic also distinguished between TD-MPC2 and a pure self-supervised-learning latent model in a cross-architecture comparison, suggesting it generalizes beyond a single training recipe.
The practical implication: teams scaling model-based RL agents may be flying blind if they rely only on reward fit. A model can look fine on conventional metrics while its internal planning dynamics have quietly broken down.
The authors frame operator-on-F as a complement to value-equivalence theory, not a replacement — which is the honest framing, though it also means practitioners now have one more diagnostic to integrate into an already crowded evaluation stack.