AI/ diffusion-models · generative-ai · model-compression · ai-research

One-Step Diffusion Models Still Denoise, Just Across Layers

Researchers found one-step diffusion models still denoise internally across layers, not time, enabling 16.6x parameter compression.

One-step image generators that skip diffusion's usual step-by-step denoising are still doing that denoising. It's just hidden inside the network's layers instead of spread across time.

Researchers studying models like MeanFlow, which compress diffusion's multi-step trajectory into a single forward pass, decoded the network's intermediate layers using its own output head. They found that the denoising computation diffusion normally performs over many sampling steps now unfolds across the depth of one pass. The effect was clearest in MeanFlow: probing shorter transport intervals revealed the network denoising and then re-noising within that single evaluation. Models without a time-indexed transport task, called drifting models, showed no equivalent pattern.

That distinction turns out to be useful, not just curious. Because the layerwise computation behaves like an actual flow, the team trained a single time-conditioned block to reproduce it, shrinking a MeanFlow SiT-L/2 model by 16.6x in parameters. Models that exhibit depthwise denoising, in other words, compress far more easily than ones that don't.

Diffusion's time axis didn't vanish when generation went one-step. It just moved, from the sampling loop into the network's own depth.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →