Speculative Decoding in vLLM on AMD GPUs
vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.
The vLLM project documents draft-and-verify speculative decoding support for AMD GPUs via ROCm, comparing native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Output-token throughput effects varied with drafting method, proposal length, model family, draft checkpoint, workload, and acceptance behavior. The post also covers how to enable each method plus practical tuning and observability considerations.
40