Unlocking Lossless Speedups in LLMs via Discrete Diffusion
Uno pairs autoregressive LLMs with lightweight diffusion weights to draw multiple tokens in parallel, delivering up to 3x lossless speedup without a draft model.
The paper introduces diffusion-augmented LLMs: autoregressive weights trained with the standard next-token objective plus lightweight diffusion weights trained via a Diffusion Distillation phase to emit multiple tokens in parallel. Psi-Spec samplers enable lossless acceleration without the separate draft model required by speculative decoding. The 8B Uno model outperforms the 26B open DiffusionGemma and proprietary Mercury 2 on agentic tool use, coding, and long-context reasoning benchmarks, with up to 3x throughput gains over the base model at all evaluated batch sizes. Code and checkpoints are released publicly.
- Decouples AR weights from lightweight diffusion weights for parallel token generation
- Psi-Spec samplers give lossless speedups without a draft model
- Up to 3x speedup over the base AR model at every evaluated batch size
- 8B Uno beats 26B DiffusionGemma and proprietary Mercury 2 on benchmarks
- Models can be trained from scratch or added to existing open-weight LLMs
- Tags
- discrete-diffusiondistillationllm-inferencelossless-accelerationopen-weightsparallel-decodingspeculative-decoding
- Products
- UnoDiffusionGemmaMercury 2
- AI models
- Uno-8B
From the original article
1 min read at huggingface.co
Large Language Models (LLMs) owe much of their success to next-token prediction (NTP), but their autoregressive (AR) structure requires slow, sequential token generation. To overcome this bottleneck, we introduce diffusion-augmented LLMs, a new class of models that defines an AR model distribution while using diffusion to draw multiple tokens in parallel from that distribution.
By Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.