PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control
PhysStream enables mid-generation interactive control of physics-grounded video via structured scene memory and velocity-increment signals, reducing motion distribution distance 33%.
PhysStream is an autoregressive physics-grounded image-to-video model that maintains structured scene memory—positional maps and object tracking maps derived online from previously generated frames—and accepts fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training runs in two stages: a bidirectional model finetuned with motion-control conditioning, then a causal autoregressive model with structured scene memory. It supports interactive mid-generation control over multi-object tabletop rigid-body scenes, reducing motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines. Human evaluators preferred it in over 85% of in-the-wild comparisons.
- Structured scene memory built online from previously generated frames
- Sparse velocity-increment signals encode physical dynamics for control
- First method supporting mid-generation control of multi-object rigid-body scenes
- FVMD cut 33%, trajectory error 12%; preferred in 85%+ human comparisons
From the original article
1 min read at huggingface.co
Interactive control for video generation is moving from coarse prompts toward fine-grained, physically meaningful manipulation of dynamic scenes. Yet existing controllable methods either require the full control schedule before generation starts, or use pixel-space signals that dictate object positions rather than physical dynamics.
By Chuhao Chen, Peter Wonka, Chaoyang Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.