Speech recognition systems that embed timestamps in their output can quietly drift off the clock during long silences — a new training method fixes that without breaking everything else.
Researchers found a specific failure mode in autoregressive automatic speech recognition (ASR) models: extended non-speech gaps cause decoded timestamp tokens to drift from reality, even when the transcript itself reads fine. They tested the problem across 15 ASR and audio-language systems using custom benchmarks, then proposed a two-stage post-training fix called REDDIT (Replay-based Distribution Editing). The method corrects timestamp targets while anchoring non-timestamp behavior to a frozen base model, then applies a short refinement pass. On Whisper-tiny, it raised long-gap accuracy from 38.7% to 95.0% mIoU and cut average alignment error from 2,752 ms to 223 ms — updating just 1.6% of model parameters on 34.9 hours of synthetic audio, with no human annotations required.
Most ASR benchmarks measure what was said, not when — so this drift goes undetected in standard evaluations. That matters for anything relying on precise timestamps: video search, captioning pipelines, and meeting summarizers that need speaker turns anchored to the recording.
The correction data is synthesized without human annotation, which is either a sign of genuine ingenuity or a reminder that self-supervised methods tend to pass clean benchmarks while stumbling on messier real-world audio.