D3D enables asynchronous speculative decoding
Original title: D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding
Standalone diffusion drafter removes sequential dependency for 5x LLM inference speedup.
D3D is a decoupled diffusion drafter combining blockwise denoising with autoregressive supervision. It drafts successor blocks during verification in one packed pass, eliminating inter-round waits. Layer pruning retains over 99% acceptance length while reducing cost, delivering up to 7.48x speedup on GSM8K with Qwen3-32B.
- 5.00x and 5.71x average speedup with Qwen3-8B and 32B
- Removes 5 of 36 layers retaining 99% acceptance
- Public code at GitHub
- Asynchronous speculative decoding
- Up to 7.48x on GSM8K with Qwen3-32B
From the original article
1 min read at huggingface.co
Target-conditioned diffusion drafters predict candidate tokens in parallel, yet each new block must wait for updated target-model hidden states. This forces drafting and verification to run sequentially, limiting speedup even though each block is generated in one pass. We introduce D3D, a standalone diffusion drafter that removes this inter-round dependency.
By Baoyou Chen, Ruiqiao Mei, Guancong Lin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.