Generating long videos with diffusion models just got significantly cheaper — at least in terms of GPU memory.
A research team has published work replacing the attention layers typically used in video diffusion models with structured state-space models (SSMs), specifically a variant called Mamba. The core problem with attention is that its memory and compute costs grow quadratically with sequence length — fine for short clips, punishing for anything longer. SSMs scale linearly instead. The team also found that bidirectional SSMs, which process sequences in both directions rather than only forward, do a better job capturing temporal context in video data, echoing findings from image generation research.
For sequences of up to 256 frames, their SSM-based models use less memory to reach the same video quality score — measured by Frechet Video Distance — as comparable attention-based models, and often outperform them at equivalent memory budgets. That matters because memory is currently one of the harder ceilings on practical long-video generation, and any architecture that relaxes it opens the door to longer, higher-resolution outputs without proportionally larger hardware.
SSMs have been circling the transformer-replacement conversation for a while now, with most of the momentum in language modeling. Seeing the gains carry into video generation — a domain with far longer effective sequence lengths — is the more interesting signal here. Whether it holds at the resolutions and durations that commercial video tools actually target remains to be seen.