VC-Attention: Nunchux AI's Training-Free Low-Bit Attention Kernel Speeds Up Video Diffusion Transformers

VC-Attention is a training-free low-bit attention method (V-Smooth + ExpCast-FP8) for video diffusion transformers, delivering 1.46-1.59x attention speedups on datacenter GPUs and up to 3.58x on workstation GPUs, with higher fidelity than SageAttention…

VC-Attention, a training-free low-bit attention framework for video diffusion transformers, appeared on Hugging Face daily papers on 2026-09-13 and was attributed to Nunchux AI in MarkTechPost coverage on 2026-09-17. It combines two components: V-Smooth, which reorders value tokens via lightweight online clustering (k-means value-token grouping with block-mean residual quantization into per-channel E4M3/NVFP4 residuals) to reduce quantization error, and ExpCast-FP8, which maps log-domain scores directly to E4M3 FP8 probability codes, replacing the FP32 softmax exponential and cast with a single fused multiply-add. Benchmarked across Wan2.2 (Wan2.2-T2V-A14B), LongCat-Video, HunyuanVideo-1.5, and MiniMax-H3, it improves fidelity over low-bit baselines and speeds attention 1.46-1.59x over BF16 FlashAttention-4 on datacenter Blackwell and Hopper GPUs (B200, B300, H200) and 2.3-3.6x on workstation cards (RTX PRO 6000, RTX 5090); specific figures include 1.59x on B200 at 8-bit and 3.58x on RTX 5090 at 4-bit for Wan2.2, with end-to-end clip generation 1.13-1.70x faster. On quality, it beats SageAttention2 by 2.3 dB PSNR on Wan2.2 at 8-bit and SageAttention3 by up to 3.6 dB at 4-bit. The kernel is training-free and composable with sparse attention, distillation, and multi-GPU execution. No public kernel release is available yet; a proprietary extension runs in Nunchux's stack.

• VC-Attention is a training-free low-bit attention framework for video diffusion transformers; it works with existing checkpoints without retraining and was introduced by Nunchux AI per MarkTechPost (2026-09-17), with the paper listed on…
• V-Smooth reorders value tokens via lightweight online clustering (k-means grouping with block-mean subtraction and per-channel E4M3/NVFP4 residual quantization) to reduce quantization error.
• ExpCast-FP8 maps log-domain scores directly to E4M3 FP8 probability codes, eliminating the FP32 softmax exponential and cast via a single fused multiply-add.
• Evaluated on Wan2.2 (Wan2.2-T2V-A14B), LongCat-Video, HunyuanVideo-1.5, and MiniMax-H3.
• Attention speedups: 1.46-1.59x over BF16 FlashAttention-4 on datacenter Blackwell and Hopper GPUs (B200, B300, H200); 2.3-3.6x on workstation cards (RTX PRO 6000, RTX 5090), including 1.59x on B200 at 8-bit and 3.58x on RTX 5090 at 4-bit…
• End-to-end clip generation is 1.13-1.70x faster (MarkTechPost cites gains up to 1.70x).
• Fidelity: outperforms SageAttention2 by 2.3 dB PSNR on Wan2.2 at 8-bit and SageAttention3 by up to 3.6 dB at 4-bit across all four tested video diffusion transformers.
• Composable with sparse attention, distillation, and multi-GPU execution.