A new technique lets diffusion models patch holes in photos without retraining for every image.
Researchers describe Amortized Inpainting with Diffusion, or AID, a method that keeps a pretrained diffusion model untouched and trains a small, reusable guidance module offline instead. That module then handles new masked images at deployment without the per-instance optimization that similar methods require. The team frames the problem as deterministic guidance with a supervised terminal objective, then uses an auxiliary Gaussian formulation to make the math tractable in high dimensions. Tested on AFHQv2, FFHQ, and ImageNet across different mask types, AID improved the trade-off between output quality and speed compared to existing fixed-backbone and amortized baselines, while adding less than one percent in trainable parameters.
That one percent number is the real story. Most inpainting systems force a choice: train a dedicated model for the task, which is expensive and inflexible, or optimize a general model separately for each image you want to fix, which is slow. AID's reusable module sidesteps both costs, which matters for any product that wants diffusion-based editing to run at consumer-app speed rather than research-lab patience.
Whether that holds up outside curated benchmarks like FFHQ and ImageNet is the usual open question with papers like this one.