PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control
PhysStream autoregressive video model enables physics-grounded mid-generation motion control, cutting trajectory error 12% and FVMD 33% versus strongest baselines.
PhysStream is an autoregressive image-to-video model that incorporates structured scene memory—positional maps and object tracking maps derived online from previously generated frames—and supports fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training proceeds in two stages: a bidirectional model finetuned with motion-control conditioning, then a causal autoregressive model with scene memory. It reduces motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines, and human evaluators prefer it in over 85% of in-the-wild comparisons.
- Autoregressive physics-grounded image-to-video synthesis with structured scene memory
- Sparse velocity-increment signals enable interactive mid-generation motion control
- 33% lower FVMD and 12% lower trajectory error than strongest baselines
From the original article
1 min read at arxiv.orgInteractive control for video generation is moving from coarse prompts toward fine-grained, physically meaningful manipulation of dynamic scenes. Yet existing controllable methods either require the full control schedule before generation starts, or use pixel-space signals that dictate object positions rather than physical dynamics.
By Chuhao Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.