Soft-MTP drops a noisy residual in multi-token prediction training
From 167M to 1.2B parameters, Soft-MTP lowers next-token perplexity and speeds speculative decoding without new parameters.
Stories per day · 30 days
2 RSS
Relatedspeculative-decoding3code1inference1language-model-training1llama1pretraining1qwen1reasoning1soft-mtp1
4 stories, newest first · one card per story
Stories tagged #multi-token-prediction
Adaptive multi-token prediction selects draft horizons online
On Llama and Qwen pairings, it was up to 4.1x faster than standard decoding and beat a fixed window in 8 of 10 settings.
ECHO folds multi-token prediction into the backbone itself
Dynamic Depth Sampling draws training depth with decaying probabilities, at half the overhead of training every depth.
Speculative Decoding in vLLM on AMD GPUs
vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.