AI/ reinforcement-learning · machine-learning · ai-research · arxiv

A Patch for an Unstable Reinforcement Learning Algorithm

RETD narrows, but does not eliminate, a stability flaw in emphatic TD learning used for off-policy reinforcement learning.

A new algorithm called RETD patches a stability flaw researchers found in emphatic temporal-difference learning, a method used to train reinforcement-learning agents on data collected under a different policy than the one being evaluated.

Emphatic TD learning, or ETD, is designed to keep off-policy training stable by reweighting updates as they happen. The researchers built a small two-state test case where ETD's math looks fine on average but the actual sequence of updates it produces fails to converge, and traced the problem to a companion term, the follow-on trace, whose variance is infinite. Their fix, regularized emphatic TD, leaves the underlying data untouched and instead stores the training signal in a separate, slowly-fading memory that applies a delayed correction. The team proves RETD converges under gradually shrinking stepsizes, shows conditional stability under fixed stepsizes, and backs both claims with paired experiments run 10,000 times.

Off-policy reinforcement learning is the backbone of systems that learn from logged data rather than live trial and error - recommender engines, some robotics pipelines - and ETD's convergence guarantees have been shakier in practice than in theory since it was introduced. RETD is a narrower claim than a full fix: the noise that makes the follow-on trace unstable is still there, unreduced, and the correction is just a way of containing where it breaks convergence.

That is a modest, honest result rather than a leap forward, and it will matter mostly to the researchers who have to make emphatic TD run reliably rather than just prove it works on paper.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →