Dream (AI model): news timeline & CVEs

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

A2D recycles autoregressive post-training weights to improve diffusion language models without extra training.

The paper shows that autoregressive post-training weight updates can be added directly to diffusion language models after AR-to-diffusion conversion, approaching the gains of diffusion-native post-training. AR and diffusion updates are nearly orthogonal in weight space but complementary, so composing them retains both. The training-free A2D method improves instruction following, mathematical reasoning, and coding on Dream, DreamReasoner, DiffuCoder, Dream-Coder, Nemotron-Labs-Diffusion, and DiffusionGemma using existing SFT and RL updates, with no added training or inference cost.

arXiv cs.AI / cs.LG / cs.CLOct 6, 10:31 UTC · 2d agoAI research1

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

Register tokens let diffusion language models like LLaDA and Dream carry reasoning state across cleared chunks, gaining up to 19.5 points on code.

Researchers propose register tokens: dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks in masked diffusion language models. After decoding and clearing a chunk, the model continues from the prompt and the carried register state instead of retaining earlier text. On LLaDA and Dream, registers outperform discrete-text carry on every benchmark, with gains up to 8.5 points on math and 19.5 points on code. Registers are especially effective for bounded code generation and can be further refined with reinforcement learning on long-horizon reasoning tasks.

Hugging Face daily papersSep 13, 20:00 UTC · 25d agoAI research1

Dream

Timeline2 articles

Appears with

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

Register Tokens for Bounded-State Reasoning in Diffusion Language Models