AI-edited images keep breaking when you look at them from a different angle — a new paper proposes using reinforcement learning to fix that.
Researchers introduced RL3DEdit, a framework that edits scenes using standard 2D diffusion models but grades its own output using a 3D foundation model called VGGT. The trick: rather than training on paired 3D-consistent editing examples — which barely exist — the system leans on VGGT's confidence maps and pose estimation errors as reward signals. Those signals tell the model when its edits look plausible from one angle but physically impossible from another, and it learns to stop doing that.
This matters because multi-view consistency is the unsolved last mile of AI scene editing. Text-to-image models can repaint a room convincingly in a single frame, but the geometry breaks the moment you move the camera. RL3DEdit sidesteps the data-scarcity problem that makes supervised fine-tuning impractical here, using verification as a training signal instead of labeled examples.
The approach borrows a pattern that has worked elsewhere: when generating the correct answer is hard but checking an answer is easy, reinforcement learning tends to outperform direct supervision. That logic drove much of the progress in LLM reasoning over the past two years, and it is now finding a home in 3D vision. The researchers say they will release the code and model, which is the part that actually matters for whether this moves beyond a conference paper.