Diffusion language models encode a hidden sense of time — and nobody told them to.
A new study shows that diffusion language models (DLMs) — a class of text generators that work by iteratively removing noise rather than predicting one word at a time — internally represent how far along the denoising process they are. The researchers found this latent timestep signal living inside the models' residual streams, extractable across multiple layers using simple probes. They also demonstrated that nudging the model along a low-dimensional subspace tied to this signal predictably shifts its confidence and output entropy. The internal representation isn't chaotic, either: it shows structured, interpretable geometry in activation space.
This matters because DLMs are not explicitly fed a timestep during inference — unlike image diffusion models, which receive that number as a direct input. The discovery suggests these models spontaneously develop an internal progress-tracking mechanism, which has real implications for interpretability and control. If you can read and steer a model's internal clock, you have a lever that wasn't on the dashboard before.
The finding lands as DLMs are being positioned as a serious rival to autoregressive models like GPT-style transformers — the current default for language tasks. Whether a hidden clock makes them more or less predictable in production is a question this paper opens rather than closes.