DNeo claims 1.3x faster speculative decoding than DFlash-2
Original title: DNeo: Your Parallel Draft Model Needs Better Target Feature Encoding
The paper says Mixture-of-Features Attention keeps drafter size and generation quality unchanged.
The DNeo paper proposes Mixture-of-Features Attention for speculative-diffusion decoding, arguing that standard target feature encoding is lossy, memory-less, and samples layers uniformly. MoFA adds a working memory of rejected proposals, with 2-tap causal convolutions, prefix-only cross-attention, and non-uniform layer selection. The authors say DNeo is 1.3x faster than DFlash-2 without changing drafter footprint or generation quality.
Unconfirmed: The 1.3x speed and quality claims are the authors' reported results, not independently confirmed here.
- The drafter proposes b tokens in one forward pass for parallel verification.
- Authors cite residual mixing as a destructive effect in standard encoding projections.
- Compared architectures named in the paper are DFlash and DSpark.
- DFlash-2 is the stated speed baseline.
From the original article
1 min read at huggingface.coSpeculative-diffusion decoding uses a small parallel drafter model to generate b draft tokens in one forward pass, which then undergo parallel verification via a larger target model. Faster inference can be achieved when multiple draft tokens pass verification simultaneously.
By Jameson Sandler, Shane Bergsma, Mostafa Elhoushi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.