Speculative Decoding in vLLM on AMD GPUs
vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.
40
vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.
Today 1.9K readers · 5K views · 3.6K reads · all time 163K views