Researchers have extended the math behind offline reinforcement learning to work with large and continuous action spaces, not just the tiny, finite ones it was built for.
Offline reinforcement learning trains a decision-making system using only a fixed batch of past data, with no live trial and error. Researchers have known since 2021 how to make that safe in theory, by having the system act pessimistically about any situation the data doesn't cover well. But the one algorithm that made this computationally practical, called PSPI, only worked with a small, finite menu of choices, and it forced the decision rule to be reverse-engineered from a separate value-estimating function rather than written directly. This paper extends the pessimism guarantees to policies written directly, over large or continuous action spaces, by isolating a snag it calls contextual coupling and solving it by linking the math to a technique called natural policy gradient.
That matters because almost nothing in the real world offers a handful of discrete choices: robots steer continuous joints, and recommendation engines rank over huge catalogs, so a theory built for toy action spaces was always a mismatch for the problems offline RL is supposed to solve. The paper also turns up a genuinely surprising result: under this framework, offline RL and imitation learning, usually treated as separate methods, turn out to share the same underlying math.
Don't expect a product here. This is pure theory, with no benchmarks or code attached, and the listing itself is a revised cross-post rather than new work. The real test is whether anyone builds a working algorithm on these proofs.