Draft refinement models revise full reasoning drafts in parallel
Original title: Breaking Diffusion: Parallel Reasoning with Draft Refinement
A 6M-parameter DRLM scores 44.4% on Countdown-5 and 93.9% on Sudoku-Extreme, beating autoregressive and diffusion baselines.
The paper introduces Draft Refinement Language Models that use one denoiser to propose parallel token revisions and score a complete draft. A cavity energy, computed in one forward pass, decides whether to keep each revision. Across Countdown, Sudoku-Extreme, and Maze-Hard, the authors report better results than autoregressive and discrete diffusion baselines.
Unconfirmed: Benchmark scores are author-reported and not independently confirmed here.
- The denoiser reconstructs every token from uniformly corrupted drafts.
- Revisions are specified directly, without a diffusion reverse transition.
- Authors report 44.4% on Countdown-5 with 6M parameters.
- The same model reaches 93.9% on Sudoku-Extreme, they report.
From the original article
1 min read at huggingface.co
Drafting an answer allows a reasoning model to reconsider each decision in the context of the others. Yet repeated diffusion denoising does not ensure that individually plausible token updates form a better answer together. We introduce Draft Refinement Language Models (DRLMs), which use a single denoiser to propose parallel revisions and evaluate a candidate draft.
By Jaesang Yu, Seoyeon Byun, Yun Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.