Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Flash-dLLM speeds diffusion LLM inference with I/O-aware KV caching and parallel draft-and-verify decoding.
Flash-dLLM is a training-free inference framework for diffusion large language models that targets slow decoding and high memory traffic. It combines an I/O-aware fused KV-cache kernel with a cache-driven draft-and-verify decoder in which the same model drafts and checks tokens. The authors report better speed and memory efficiency than prior accelerators, including 5.1× and 11.0× speedups over Elastic-Cache on GSM8K and HumanEval.
- Flash-dLLM is a training-free accelerator for diffusion LLMs.
- A fused KV-cache kernel reduces redundant GPU memory movement.
- The same model drafts and verifies tokens without an auxiliary drafter.
- Reported speedups are 5.1× on GSM8K and 11.0× on HumanEval.
From the original article
1 min read at arxiv.orgDiffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabling non-autoregressive text generation. However, their practical deployment remains limited by inefficient inference, largely due to the absence of effective Key-Value (KV) caching and scalable parallel decoding mechanisms.
By Quan Nguyen-Tri · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.