A theory of adaptive behavior borrowed from neuroscience just got translated into the language every reinforcement-learning researcher already speaks.
A new paper reformulates Active Inference (AIF) - the theory that behavior minimizes "expected free energy," a score blending curiosity about the world with progress toward a goal - as policy optimization over closed-loop control policies. The authors show this version of AIF is a convex Markov decision process (MDP), a well-understood class of problem with existing solvers and convergence guarantees. That reframing also clarifies what AIF's reward-prediction-error updates actually do: they carry free-energy gradients backward through time, not upward through a cortical hierarchy as some neuroscience accounts suggest. The paper further shows that when an agent learns its world model alongside its policy, the setup takes on the structure of performative reinforcement learning, where the environment's effective payoff shifts because the agent is acting on it.
This is a translation, not a discovery about brains, but it's a useful one. It moves AIF out of philosophy-of-mind seminars and into a framework where reinforcement-learning researchers can apply existing convex-optimization tools, solvers, and proofs instead of building bespoke variational math from scratch.
It doesn't settle whether Active Inference is a good model of the brain - it just makes the theory legible to people who don't read predictive-coding papers for fun.