SpecFold: Folding Multi-Branch Redundancy for Faster Speculative Decoding in Diffusion Language Models
SpecFold reuses multi-branch computation to speed diffusion-LLM speculative decoding by up to 1.99×.
Diffusion language models generate text by iterative block denoising, and multi-branch speculative decoding verifies a main branch plus drafts in one forward pass. SpecFold exploits highly similar hidden states across draft branches with token-level residual gating and selective reuse of parent attention and FFN computation, implemented in a Triton kernel. Across two DLLM families, five models, and five benchmarks it reaches up to 1.64× throughput over Spiffy and 1.99× over vanilla decoding with comparable task performance. The method is orthogonal to temporal caching and compatible with existing speculation strategies.
- Exploits multi-branch hidden-state similarity during speculative verification
- Reuses parent compute via residual gating, folded attention, and FFN
- Triton kernel turns fine-grained reuse into sparse multi-branch execution
- Up to 1.64× over Spiffy and 1.99× over vanilla decoding
- Orthogonal to temporal caching and existing DLLM speculation strategies
From the original article
1 min read at huggingface.co
Diffusion large language models (DLLMs) generate text through iterative block denoising, and multi-branch speculative decoding accelerates this process by verifying a main branch together with multiple draft branches in a single forward pass.
By Chung-En Ho, Weiyu Sun, Cheng-Jhih Shih · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.