Schedule optimization for tau-leaping in masked discrete diffusion
A theory paper derives optimal denoising schedules for tau-leaping samplers in masked discrete diffusion models.
The paper analyzes factorization error when tau-leaping samplers for masked discrete diffusion reveal coordinates in parallel and replace the joint conditional with a product distribution. It expresses that error through a dependence density, develops estimators, and characterizes a unique optimizer for the finite-step schedule under a monotonicity condition. When the dependence profile stays strictly positive, optimizing smooth schedules can improve the leading constant but not the N/K scaling; degenerating profiles can improve the asymptotic order relative to a uniform schedule.
- Tau-leaping incurs factorization error even with perfect predictors.
- Error is represented by a dependence-density profile over revealed coordinates.
- A unique finite-step schedule optimizer is characterized under monotonicity.
- Strictly positive profiles keep N/K scaling; degenerating profiles can improve it.
From the original article
1 min read at arxiv.orgMasked discrete diffusion models are commonly accelerated using the so-called tau-leaping discretization method, which reveals several coordinates in parallel at each sampling step. The sampler replaces the joint conditional law of each revealed block by a product distribution, incurring a factorization error $\varepsilon_\text{fact}$ present even with perfectly learned predictors.
By Cecilia Secchi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.