Diffusion language models vary compute cost by task and language
Original title: Diffusion Language Models Change Speed Across Tasks and Languages
Weighting math answers by decoding speed beat majority voting by up to 4.2 points at no extra cost.
A paper analyzes two diffusion language models across 10 tasks and finds forward passes per token depend heavily on the task. Copying or light edits are cheap, while open-ended writing is costly; novelty relative to the input and rare words explain much of the gap. Highly tokenized languages need more passes per token. On the same math problem, wrong answers take more passes than correct ones, and speed-weighted voting beat majority voting by up to 4.2 points.
- In DiffusionGemma, story writing needs more than three times the passes of grammar correction.
- A final answer is cheap when it repeats the reasoning trace and costly when it adds new text.
- Two outputs of equal token length can still differ sharply in generation cost.
- The authors say diffusion’s compute advantage depends on how determined the output already is.
From the original article
1 min read at huggingface.co
Diffusion language models can write several tokens in one forward pass, and they decide during generation how many passes to spend. Two outputs of the same token length can therefore vary a lot in cost. To understand the underlying factors of this cost variation, we analyze two diffusion models across 10 tasks.
By Mohsen Fayyaz, Ali Modarressi, Daniel Israel · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.