Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation

Video DeltaNet combines local Softmax attention with linear memory for video, cutting DiT denoising of a 14.3-second 768p video to 6.70 seconds on eight NVIDIA B200 GPUs — a 14.5x speedup over the 50-step dense H3 baseline.

Video DeltaNet (VDN) is a video-native hybrid attention architecture that pairs local Softmax attention with a bidirectional linear memory branch for long-range context, introducing Video Delta Attention, which updates the linear memory once per frame while jointly incorporating its spatial tokens. It is instantiated on MiniMax H3, applying the hybrid to video-to-video interactions while retaining Softmax attention for interactions involving text or audio. A staged teacher-alignment recipe progressively introduces the new pathway into pretrained models. With eight-step distillation and an optimized SGLang serving stack, VDN-H3 completes DiT denoising of a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs, a 14.5x speedup over the 50-step dense H3 baseline on identical GPU count. The work surfaced via Hugging Face daily papers on 2026-09-16 and an arXiv listing (cs.AI / cs.LG / cs.CL) on 2026-09-17; the two sources agree on all figures.

• Video DeltaNet (VDN) pairs local Softmax attention with a bidirectional linear memory branch for long-range context
• Video Delta Attention updates the linear memory once per frame while jointly incorporating its spatial tokens
• VDN is instantiated on MiniMax H3; the hybrid is applied to video-to-video interactions while Softmax attention is retained for text or audio interactions
• A staged teacher-alignment recipe progressively introduces the new pathway into pretrained models
• Uses eight-step distillation and an optimized SGLang serving stack
• VDN-H3 completes DiT denoising of a 14.3-second, 768p video in 6.70 seconds on eight NVIDIA B200 GPUs
• This is a 14.5x speedup over the 50-step dense H3 baseline on identical GPU count
• Reported via Hugging Face daily papers (2026-09-16) and arXiv cs.AI / cs.LG / cs.CL (2026-09-17); sources agree on all figures