Self-Conditioning Enables Deep Sequential Reasoning in Flow Matching
Researchers show self-conditioning lets flow-matching models gain deeper visual reasoning from additional sampling steps.
A paper argues that vanilla flow-matching samplers do not gain sequential reasoning power from extra sampling steps when outputs meet mild regularity conditions, because their trajectories can be simulated by a constant number of sequential rounds of parallel model evaluations. The authors show that self-conditioning, which feeds the previous step’s estimate of the final image back into the model, removes that bottleneck. Controlled visual-reasoning experiments find higher accuracy and greater reasoning depth than vanilla flow matching, with performance improving as sampling steps increase. They prove a constant-depth backbone with self-conditioning can match the expressive power of space-bounded Turing machines.
- Vanilla flow matching can be simulated with a constant number of sequential rounds.
- Extra sampling steps do not deepen a constant-depth transformer under stated conditions.
- Self-conditioning passes the previous estimate of the final image into the next step.
- Experiments show higher accuracy and reasoning depth that improve with more steps.
- With self-conditioning, a constant-depth model matches space-bounded Turing-machine power.
From the original article
1 min read at huggingface.co
Recent work has explored visual reasoning through image generation, where models solve visual-reasoning problems by directly generating solution images. Intuitively, iterative samplers such as flow matching are well suited to this task, since each sampling step may resemble one reasoning step. However, in this work, from a circuit complexity perspective, we show that this intuition is misleading.
By Zeyang Zhang, Linxiang Gao, Meiqi Gu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.