Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Flash-dLLM accelerates diffusion LLM inference with I/O-aware KV caching, up to 11 times faster than Elastic-Cache.
Flash-dLLM is a training-free inference framework for diffusion large language models that treats GPU memory I/O as the main bottleneck when KV caching and parallel decoding are used together. An I/O-aware fused KV-cache kernel reduces redundant memory traffic, and a cache-driven draft-and-verify strategy lets the same model draft and check tokens without an auxiliary drafter. The authors report better speed and memory use than prior dLLM accelerators, including 5.1 times and 11.0 times speedups over Elastic-Cache on GSM8K and HumanEval, while preserving generation quality and scaling to longer sequences and larger batches.
- Training-free inference acceleration for diffusion large language models.
- A fused KV-cache kernel reduces redundant GPU memory movement.
- The same model drafts and verifies tokens without an auxiliary model.
- Reports 5.1 times and 11 times speedups over Elastic-Cache on GSM8K and HumanEval.
Full article209 words · extracted from huggingface.co · click to collapse
Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabling non-autoregressive text generation. However, their practical deployment remains limited by inefficient inference, largely due to the absence of effective Key-Value (KV) caching and scalable parallel decoding mechanisms. Existing acceleration methods typically study KV caching and parallel decoding in isolation, overlooking the I/O bottlenecks that arise when cache reuse and parallel token verification are jointly applied. In this work, we introduce Flash-dLLM, a training-free inference acceleration framework for fast and memory-efficient dLLMs. Flash-dLLM first identifies GPU memory I/O as a dominant bottleneck in KV-cache-enabled dLLM inference and addresses it with an I/O-aware fused KV-cache kernel that reduces redundant memory movement. Building on this optimized cache mechanism, Flash-dLLM further proposes an efficient KV-cache-driven draft-and-verify decoding strategy, where the dLLM itself serves as both drafter and verifier without requiring an auxiliary model. This unified design enables faster decoding while preserving generation quality and improving scalability to longer sequences and larger batch size. Extensive experiments on mathematical reasoning and code-generation benchmarks demonstrate that Flash-dLLM consistently outperforms existing state-of-the-art dLLM acceleration methods in both inference speed and memory efficiency. In particular, it achieves 5.1times and 11.0times speedups over prior strongest baseline Elastic-Cache on GSM8K and HumanEval, respectively.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.26796