D3D enables asynchronous speculative decoding
Original title: D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding
D3D drafts during verification without target hidden states for 5.71x speedup.
D3D is a standalone diffusion drafter that removes inter-round dependencies. It combines blockwise denoising with autoregressive supervision to achieve long accepted prefixes. Removing five of 36 layers retains over 99% of mean acceptance length. Across seven benchmarks it averages 5.00x and 5.71x speedup.
- D3D prunes 5 layers from 36
- 99% acceptance length retained
- 5.00x average speedup
- 7.48x on GSM8K
- Code at github.com/fudan-generative-vision/D3D
From the original article
1 min read at arxiv.orgTarget-conditioned diffusion drafters predict candidate tokens in parallel, yet each new block must wait for updated target-model hidden states. This forces drafting and verification to run sequentially, limiting speedup even though each block is generated in one pass. We introduce D3D, a standalone diffusion drafter that removes this inter-round dependency.
By Baoyou Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.