Frequency-aware remasking lifts diffusion language model test-time scaling
MATH-500 gains reach 3.73 points for LLaDA-8B-Instruct and 7.33 points for Dream-7B-Instruct.
Stories per day · 30 days
2 RSS
Relatedinference4llada4openwebtext3dream2gsm8k2looping2training-efficiency2credit-assignment1decoding1diffusiongemma1fineweb-edu1hidden-messages1humaneval1langdit1langflow1language-modeling1language-models1latent-communication1
25 stories, newest first · one card per story
Stories tagged #diffusion-language-models
TDLM reuses past denoiser states to speed diffusion language models
On LLaDA the added modules are about 0.84% of parameters and generation is up to 7.15 times faster.
dSpark distills unmasking schedules to speed diffusion language models
On GSM8K the paper reports 10.12× and 5.12× speedups on LLaDA-8B-Instruct and Dream-7B-Instruct while keeping task quality.
DTCA assigns token-level credit for RL on diffusion language models
On LLaDA-8B-Instruct, authors report gains of up to 9.0 points on MATH500 and 5.3 on GSM8K.
SPOT predicts commit order to accelerate diffusion language decoding
On SDAR-8B, SPOT cuts target passes by up to 6.8× and reaches up to 3.4× standard throughput.
Study isolates what improves learned unmasking in diffusion language models
An 856K-parameter policy on LLaDA-8B-Instruct leads the accuracy–NFE frontier on GSM8K and largely transfers to Dream-7B-Instruct.
T2Quant allocates precision by task and timestep in diffusion LMs
On three diffusion language models and six benchmarks, the method raises average accuracy by up to 6.7 points at a fixed bit budget.
LAW bases language modeling entirely on self-conditioning
After TinyGSM training, LAW solves 42.2% of GSM8K at 1024 NFE versus 34.5% for uniform loopholing discrete diffusion.
Looping cuts training FLOPs for uniform diffusion language models
Looped UDLMs match a roughly 0.5-trillion-token baseline with up to 2.8x fewer FLOPs on OpenWebText and 3.3x fewer on FineWeb-Edu.
Looped uniform diffusion language models cut training FLOPs
Selective looping matches a non-looped UDLM's validation NLL with up to 3.3x fewer training FLOPs.
Diffusion LM agents pass usable hidden messages in latent states
On QASC, matched messages reach 80.8%, while a wrong-example message lowers accuracy to 46.8%.
Diffusion language models vary compute cost by task and language
Weighting math answers by decoding speed beat majority voting by up to 4.2 points at no extra cost.
Designing Transformers for Continuous Diffusion Language Models
LangDiT redesigns Transformers for continuous diffusion LMs, cutting OpenWebText generation perplexity to 11.33–21.72.
On the Design Space of Continuous Diffusion Language Models
A study of continuous diffusion language models introduces DoubleCat and SemiCat, matching discrete diffusion on TinyGSM.
Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Next-Concept Modeling generates text coarse-to-fine in parallel, up to 31 times faster than diffusion language models.
Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
Hierarchical continuous diffusion models jointly denoise tokens and clusters, improving generative perplexity by over 20 points.
Enhancing Diffusion Language Models with Autoregressive Post-Training Weights
A2D recycles autoregressive post-training weights to improve diffusion language models without extra training.
Noise Out, Bias In: Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steering
A closed-loop steering attack injects targeted demographic bias into diffusion language models during denoising.
Representation-Space MMD for Diffusion Language Models
Representation-space MMD post-training improves diffusion language models without full sampling trajectories.
Pivot-SD: Efficient Self-Distillation for Masked Diffusion Language Models
Pivot-SD self-distills masked diffusion models by training only high-impact commitments, improving LLaDA-8B on math and code.
Distribution Matching Distillation for Continuous Diffusion Language Models
Simplex-DMD distills continuous diffusion language models to 4 evaluations with perplexity 45.6.
E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models
E-MoE adds a discrete MoE latent to diffusion language models and improves few-step generation.
dQwen3.5: Hybrid-Attention Diffusion Language Models
Researchers adapt Qwen3.5 hybrid-attention backbones (0.8B-9B) into diffusion language models, reaching training loss in half the tokens versus full attention.
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Register tokens let diffusion language models like LLaDA and Dream carry reasoning state across cleared chunks, gaining up to 19.5 points on code.