DLoop: Looped Speculative Decoding
DLoop loops speculative decoding so draft models propose more tokens before one joint verification, cutting target-model passes.
DLoop is a looped speculative-decoding method that keeps drafting while the draft model is confident and verifies accumulated tokens together. Loop-aware training keeps the draft model reliable on unverified tokens by exposing it to its own hidden states. Across EAGLE-3, DFlash, Domino, DSpark, and multi-token prediction, it improves wall-clock speedup by 5 to 41 percent without changing the decoded output. Code is planned at github.com/naver-ai/DLoop.
- DLoop runs several drafting stages before one joint verification.
- Loop-aware training exposes the draft model to its own unverified hidden states.
- Wall-clock speedup rises 5–41% across EAGLE-3, DFlash, Domino, DSpark, and multi-token prediction.
- Decoding stays lossless while fewer target-model forward passes are needed.
- Code is planned at github.com/naver-ai/DLoop.
- Organizations
- Naver AI
From the original article
1 min read at huggingface.co
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all tokens produced in a drafting stage.
By Geonmo Gu, Byeongho Heo, HeeJae Jun · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.