DScout speeds diffusion-drafted speculative decoding on Qwen3
Original title: Factorization-Guided Training and Inference for Diffusion-Drafted Speculative Decoding
Mean speedups versus autoregressive decoding are 3.84 times on Qwen3-4B and 4.14 times on Qwen3-8B.
DScout uses two factorizations to train and run diffusion drafters so higher acceptance does not raise latency enough to erase throughput gains. Training aligns supervision with prefix acceptance; inference chooses conditioning depth, drafter depth, and block size from the ratio of mean accepted tokens to per-round latency. Across six benchmarks the authors report large speedups on Qwen3-4B and Qwen3-8B.
Unconfirmed: Speedups are author-reported against their chosen baseline; independent benchmarks are not stated.
- Mean speedup over the stated baseline rises 9.1% and 7.1%.
- An inference policy adds up to 9.75% mean speedup over a fixed setup.
- Early rejections can waste later correct draft predictions.
- Results cover multiple models and concurrency settings.
- Products
- DScout
From the original article
1 min read at huggingface.co
Speculative decoding accelerates large language model inference through parallel verification of tokens proposed by a drafter. Diffusion drafters further parallelize proposal generation, but greater proposal capacity or higher mean accepted tokens (MAT) need not yield higher throughput.
By Jiakun Fan, Min Xiao, Qiang Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.