DTCA assigns token-level credit for RL on diffusion language models
Original title: DTCA: Denoising Token Credit Assignment for Reinforcement Learning in Diffusion Language Models
On LLaDA-8B-Instruct, authors report gains of up to 9.0 points on MATH500 and 5.3 on GSM8K.
Researchers propose Denoising Token Credit Assignment (DTCA) for reinforcement learning on diffusion language models, which generate text by iterative denoising. DTCA estimates each token’s contribution from confidence gains on remaining masked positions, groups signals by part of speech, and selects informative denoising timesteps. Learning signals are also calibrated using token-level entropy changes from policy updates. Experiments use LLaDA-8B-Instruct and report better mathematical reasoning across generation budgets.
Unconfirmed: The MATH500 and GSM8K gains are the authors’ reported results, not independently verified.
- Credit is measured from confidence gains after a token is committed.
- Signals are grouped by part-of-speech categories.
- Entropy changes from policy updates calibrate token-level learning signals.
- Reported gains reach 9.0 points on MATH500 and 5.3 on GSM8K.
- AI models
- LLaDA-8B-Instruct
From the original article
1 min read at huggingface.co
Diffusion language models (DLMs) generate text through iterative denoising, enabling bidirectional context modeling and parallel token refinement. Recent studies have applied reinforcement learning (RL) to DLMs and improved their reasoning performance.
By Hongze Chen, Miao Pan, Lizhou Liang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.