Researchers have built an interactive framework that maps how diffusion models distribute attention across every generation step — token by token, from initial noise to finished image.
The tool, described in a new paper, targets a specific blind spot in how engineers study text-to-image models like Stable Diffusion. Most existing pipelines collapse attention data into aggregated heat maps or single scalar values, stripping out the temporal information that shows how a model shifts focus as it refines an image. This framework instead indexes attention maps by step, tracks how concentrated that attention is over time, and surfaces spatial relationships — all inside a linked, interactive interface. The authors validated it on a structured benchmark of 60 prompts, finding interpretable recurring patterns in how these models build up semantic structure.
Interpretability work on image-generation models has lagged well behind the equivalent effort on language models, and that gap is widening as these systems get embedded in commercial products. A tool that lets a human observer trace exactly when and where a model commits to rendering a concept gives researchers a concrete handle on failure modes — rather than guessing from output alone.
The benchmark is 60 prompts, which is small; whether those interpretable patterns hold across more diverse inputs is a question the paper does not answer.