Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

Flash-dLLM, a training-free inference framework for diffusion LLMs, uses an I/O-aware fused KV-cache kernel and cache-driven draft-and-verify decoding to cut GPU memory traffic, reporting 5.1x and 11.0x speedups over Elastic-Cache on GSM8K and HumanEval.

Flash-dLLM is a training-free inference acceleration framework for diffusion large language models (dLLMs) that identifies GPU memory I/O as the main bottleneck when KV caching and parallel decoding are combined. It addresses this with an I/O-aware fused KV-cache kernel that reduces redundant GPU memory movement, paired with a cache-driven draft-and-verify parallel decoding strategy in which the same model both drafts and verifies tokens, without requiring an auxiliary drafter. The authors report better speed and memory efficiency than prior dLLM accelerators, including 5.1x and 11.0x speedups over Elastic-Cache on GSM8K and HumanEval respectively, while preserving generation quality and scaling to longer sequences and larger batches. The paper appeared on Hugging Face daily papers (2026-09-21) and in arXiv listings for cs.AI, cs.LG, and cs.CL (2026-09-22); the two sources are fully consistent, with no disagreements.

• Flash-dLLM is a training-free inference framework for diffusion large language models (dLLMs).
• It treats GPU memory I/O as the main bottleneck when KV caching and parallel decoding are used together.
• An I/O-aware fused KV-cache kernel reduces redundant GPU memory movement.
• A cache-driven draft-and-verify strategy lets the same model draft and verify tokens without an auxiliary drafter.
• Reported speedups over Elastic-Cache: 5.1x on GSM8K and 11.0x on HumanEval.
• The framework preserves generation quality and scales to longer sequences and larger batches.
• Published via Hugging Face daily papers on 2026-09-21 and arXiv (cs.AI, cs.LG, cs.CL) on 2026-09-22.