AB-Attention approximates dense video attention at high sparsity
Original title: AB-Attention: Accurate Block Attention for Accelerating Video Diffusion Transformers
On Wan2.1 and HunyuanVideo the authors report skipping 89–92% of attention with up to 2.23× speedup.
Researchers propose Accurate Block Attention (AB-Attention), a training-free method that approximates dense 3D attention in video diffusion transformers. Intra-block clustering, inclusion-probability sampling, and a finer key/value gather are meant to avoid dropping useful keys under aggressive sparsity. The authors report that on Wan2.1 and HunyuanVideo it skips 89–92% of attention while baselines compute 1.8–3.6 times as much, with up to 4.4 dB higher PSNR and up to 2.23× end-to-end speedup. Code is at the InfRec-Cardinal GitHub repository.
Unconfirmed: Speed and quality figures are the authors' reported results, not independently verified.
- Training-free; no token permutation is required for the selector.
- Authors report up to 4.4 dB PSNR over the strongest baseline.
- Kernel packs sub-tile key/value blocks into full tensor-core tiles.
- Evaluated on Wan2.1 and HunyuanVideo.
- Products
- Wan2.1HunyuanVideo
- AI models
- AB-Attention
From the original article
1 min read at huggingface.co
Video diffusion transformers generate high-quality videos, but the quadratic cost of 3D full attention dominates their inference. Dynamic block-sparse attention reduces this cost by computing exact attention only on key/value blocks selected from current activations.
By Bin Xu, Mo Zhang, Zhengfeiyang Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.