Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Video DeltaNet combines local Softmax attention with linear memory, enabling 14.5x faster 768p video generation on eight NVIDIA B200 GPUs.
Video DeltaNet (VDN) is a video-native hybrid attention architecture pairing local Softmax attention with bidirectional linear memory for long-range context, introducing Video Delta Attention that updates memory once per frame with spatial tokens. It is instantiated on MiniMax H3, applying the hybrid to video-to-video interactions while retaining Softmax for text and audio. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 completes DiT denoising for a 14.3-second 768p video in 6.70 seconds on eight NVIDIA B200 GPUs, a 14.5x speedup over the 50-step dense H3 baseline.
- Hybrid design pairs local Softmax attention with bidirectional linear memory
- Video Delta Attention updates memory once per frame with spatial tokens
- Staged teacher-alignment recipe introduces the pathway into pretrained models
- 14.5x speedup over dense H3 baseline on identical GPU count
Full article175 words · extracted from huggingface.co · click to collapse
Video diffusion models repeatedly process long spatiotemporal token sequences during denoising, making attention a major computational bottleneck. Linear attention offers an appealing alternative and has been widely adopted in recent large language models, but directly applying it to video models often fails to preserve the fine-grained interactions required for high-quality generation. We present Video DeltaNet (VDN), which combines local Softmax attention with bidirectional linear memory for long-range video context. Its linear branch introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens. Separate output projections and learnable gates calibrate the two branches, while a staged teacher-alignment recipe progressively introduces the new pathway into pretrained models. We instantiate VDN on MiniMax H3, applying the hybrid to video-to-video interactions while retaining Softmax for interactions involving text or audio. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 completes DiT denoising for a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs, corresponding to a 14.5x speedup over the 50-step dense H3 baseline on the same GPU count.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.20744