First-Order Stationarity of Reverse Diffusions
SDE reverse flows of Langevin diffusions contract Fisher divergence when the noising potential is strongly convex.
The paper develops a first-order theory linking optimization and sampling for diffusion models. It shows that SDE-based reverse-time flows of overdamped and underdamped Langevin diffusions contract relative Fisher divergences at explicit exponential rates when the forward process's stationary potential is strongly convex, a condition on the chosen noising process rather than the data. The authors say this advantage is absent from ODE-based reverse processes. They also prove averaged first-order stationarity bounds for discretized overdamped and underdamped samplers; the convexity-free certificate is local and guarantees score consistency, not global mode weights.
- Reverse SDE flows contract relative Fisher divergence under strong convexity.
- Convexity is required of the noising potential, not the data.
- ODE-based reverse processes lack this contraction property.
- Discretized samplers receive averaged first-order stationarity bounds.
- Those bounds certify local score consistency, not global mode weights.
From the original article
1 min read at arxiv.orgRecent literature has shown a strong connection between optimization and sampling. We develop the corresponding first-order theory for diffusion models.
By Zhifeng Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.