Researchers have a new fix for the instability that plagues reinforcement learning on image-generating AI models.
In a preprint posted to arXiv ("AdvantageFlow: Regularized Advantage-Weighted RL in Flow Models," arXiv:2605.26013), the authors introduce AdvantageFlow, an RL algorithm built for rectified flow models - the architecture behind tools like Stable Diffusion 3.5 Medium and FLUX.1. The method trains on the forward, noise-to-image process rather than reversing it step by step. It weights each training update by how much better an action performed than expected, its "advantage," then regularizes that update against the policy that generated the rollout. That regularization term, the paper says, turns a normally unstable optimization problem into a convex one and cuts down variance.
RL fine-tuning of diffusion and flow models is notoriously twitchy - push too hard on a reward signal and outputs collapse into reward-hacking artifacts instead of genuinely better images. A regularization trick that stabilizes training without hand-tuned schedules matters more to the people building these models than to anyone browsing a gallery of outputs, but it is exactly the kind of gap that decides whether RL techniques make it out of research papers and into production fine-tuning pipelines.
The paper reports gains over both forward-process and reverse-process RL baselines on Stable Diffusion 3.5 Medium and FLUX.1, but it's one team's benchmark with no independent replication yet - worth watching, not worth betting a training run on just yet.