ProAR: Learning Prospective Reasoning with Autoregressive Video Models
ProAR makes autoregressive video generation goal-oriented via predicted goal frames, improving visual reasoning using only 25% of standard training steps.
ProAR transforms autoregressive video generation into a goal-oriented reasoning process by adding goal-frame prediction through an asymmetric attention mask and future representation self-alignment for short-range transitions. The two mechanisms combine sparse target supervision with dense step-wise guidance. ProAR surpasses fully trained standard AR baselines while using only 25% of the training steps, and shows promise on embodied reasoning tasks.
- Goal-frame prediction anchored via asymmetric attention mask guides intermediate state generation
- Future representation self-alignment anticipates upcoming temporal dynamics
- Beats fully trained AR baselines with only 25% of training steps
- Shows applicability to embodied reasoning tasks
From the original article
1 min read at huggingface.co
Autoregressive (AR) video models excel at causal generation, but their reliance on next-chunk prediction confines them to a short-sighted, reactive paradigm. This limitation is particularly consequential for reasoning-oriented generation, where achieving a target outcome through valid intermediate states matters more than local visual plausibility.
By Linghui Shen, Tinghui Zhu, Sheng Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.