SPOT predicts commit order to accelerate diffusion language decoding
Original title: On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction
On SDAR-8B, SPOT cuts target passes by up to 6.8× and reaches up to 3.4× standard throughput.
The paper argues speculative decoding fails to transfer directly to diffusion language models because they must choose both tokens and positions. SPOT, a 0.62M-parameter model, reads only the target's output distributions, predicts commit order, and proposes a tree the target verifies in one pass. Coupled-noise match shares Gumbel noise so verification stays exact at any temperature. SPOT-Sets drafts whole commit sets; the learned drafter transfers across six DLMs, and with light calibration SPOT-Sets does too.
- Previous-pass predictions already hold the right token at over 90% of positions.
- SPOT-Sets raises SDAR-8B throughput 59% on average over threshold decoding alone.
- The drafter transfers across six different diffusion language models.
- Results match or beat calibrated training-free baselines at T=0 and T=1.
From the original article
1 min read at huggingface.co
Diffusion language models (DLMs) promise parallel text generation, but in practice face an unresolved speed-quality trade-off. Exact decoding fills in only one masked position per model pass, while faster multi-token decoding schemes are lossy. Speculative decoding is the standard lossless acceleration for autoregressive models.
By Eliezer Shlomi, Eliron Rahimi, Avi Mendelson · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.