Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning
DBTM achieves one-step text generation via a time-independent transport map trained directly from data, removing pretrained teacher distillation.
Discrete Beckmann Transport Models (DBTM) build a time-independent flow whose autonomous transport map provably carries any point in ambient space to a fixed point on simplex vertices in a single step. The fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, eliminating the need for a teacher flow, distillation, and time conditioning. A partial-context interpolant extension turns additional function evaluations into refinement steps rather than ODE integration steps. On language modeling and reasoning tasks, DBTM's one- and few-step generation improves quality and accuracy over discrete diffusion and continuous flow baselines.
- Autonomous transport map reaches simplex vertex fixed point in a single step
- Trained by minimizing conservation-equation residual directly; no teacher flow or distillation required
- Partial-context interpolant makes extra evaluations refinement steps, not ODE steps
- Outperforms discrete diffusion and continuous flow baselines in one- and few-step generation
Full article184 words · extracted from arxiv.org · click to collapse
Discrete diffusion and flow models are a promising alternative to autoregressive language models, but compressing many-step sampling into fewer steps typically requires distilling a pretrained teacher model. This caps the student at the teacher's quality and requires a costly two-stage training pipeline. We introduce Discrete Beckmann Transport Models (DBTM), built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the vertices of the simplex in a single step. We show that this fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, removing the requirement for a teacher flow and time conditioning. Under this construction, a partially trained map corresponds to the flow truncated at finite time, so generation reduces to iterating one map until it reaches a fixed point. We further extend the map to a partial-context interpolant where additional function evaluations act as refinement steps rather than ODE integration steps. On language modeling and reasoning tasks, DBTM enables one- and few-step generation that improves quality and accuracy over discrete diffusion and continuous flow baselines.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.15903