A new paper proves, for the first time, that an AI agent can be asymptotically near-optimal in general reinforcement learning without ever building an explicit model of its environment.
Researchers introduced AIQI (Universal AI with Q-Induction), a theoretical agent that performs universal induction over distributional action-value functions rather than over environments or policies, as prior agents did. The distinction matters: every previously established optimal agent in general RL - including AIXI, the field's canonical benchmark - required maintaining an explicit environment model. AIQI sidesteps that entirely. Under the standard "grain of truth" assumption (the true environment must be representable within the agent's hypothesis space), the authors prove AIQI is strongly asymptotically ε-optimal and asymptotically ε-Bayes-optimal. As a secondary result, they apply their new proof techniques to show that Self-AIXI achieves the same guarantees without the ad-hoc assumptions earlier analyses depended on.
The model-based versus model-free split is one of reinforcement learning's oldest structural debates, and the theoretical optimality literature had always landed on the model-based side - the assumption being that you simply needed a world model to achieve the best possible behavior at scale. This proof breaks that assumption at the foundational level, opening a new branch of universal agent research that didn't exist before yesterday.
The caveats are real: AIQI is computationally intractable by design, joining AIXI in the category of agents that are optimal in principle and impossible to run in practice. Theoretical results like this typically take decades to influence engineering - if they ever do.