The Lattice of Transition Laws
Researchers model diffusion and autoregressive decoding as lattice paths and predict schedule cost from data geometry.
The paper treats diffusion, autoregression, and hybrids as paths on a single corruption lattice, defining a schedule's cost as dependence discarded by parallel steps. It argues the fewest zero-cost steps equal the data graph's treedepth: logarithmic in sequence length and linear in grid side length. With fewer steps, every schedule pays a positive cost, ranked before decoding via a pairwise-dependence kernel estimated from pretrained weights. Predictions are checked on text, image, and video generation, and code is released on GitHub.
- Diffusion, AR, and hybrids are paths on one corruption lattice.
- Zero-cost step count equals graph treedepth for Markov data.
- Treedepth is logarithmic in sequence length, linear in grid side.
- Pairwise-dependence kernel ranks schedules before decoding.
- Rankings mostly verified on text, image, and video generation.
From the original article
1 min read at huggingface.co
Diffusion and autoregression (AR) have long been seen as different categories of generative models, with diffusion specialising in continuous fields and AR specialising in discrete tokens. Recent work seeks to combine the advantages of the two models, and each hybrid fixes its decoding schedule by design.
By T. Y. Tsui, Jiatao Gu, Lingjie Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.