ALLOT allocates block-sparse attention for video diffusion transformers
Original title: Sparsity Should Be ALLOTted by Output: Training-Free Allocation of Block-Sparse Attention for Video Diffusion Transformers
Training-free ALLOT gains 11.8 dB PSNR on HunyuanVideo with XAttention at equal or shorter runtime.
The paper says full 3D attention is costly in video Diffusion Transformers and that existing block-sparse methods allocate sparsity poorly. ALLOT estimates each layer-step cell's effect on final-video error through cell probing and trajectory calibration, then sets densities under a measured latency budget without retraining. It keeps existing selection strategies and kernels. Across Wan2.1, Wan2.2, and HunyuanVideo, it outperformed or tied the default in all nine reported settings.
- Sensitive layer-step pairs stayed nearly unchanged across inputs.
- ALLOT replaces only the allocation component, not kernels or selection.
- The 11.8 dB PSNR gain is reported with XAttention on HunyuanVideo.
- Experiments cover four additional published pipelines and nine settings.
From the original article
1 min read at huggingface.co
Full 3D attention is costly in video Diffusion Transformers. Block-sparse attention reduces this cost by computing only selected attention blocks. However, existing methods either overlook the potential to optimize sparsity allocation or rely on heuristic rules or local metrics to guide it.
By Tianze Xia, Xiaoling Zhou, Ruya Lan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.