PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

PhysStream, an autoregressive physics-grounded image-to-video model, uses structured scene memory and sparse velocity-increment signals to enable interactive mid-generation motion control, cutting FVMD by 33% and trajectory error by 12% versus the strongest…

PhysStream is an autoregressive physics-grounded image-to-video model that maintains structured scene memory—positional maps and object tracking maps derived online from previously generated frames—and accepts fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training runs in two stages: a bidirectional model finetuned with motion-control conditioning, followed by a causal autoregressive model with structured scene memory. The model supports interactive mid-generation control over multi-object tabletop rigid-body scenes—one report describes it as the first method to do so—and reduces motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines. Human evaluators preferred it in over 85% of in-the-wild comparisons. The work surfaced on Hugging Face daily papers on 2026-09-14 and on arXiv (cs.AI/cs.LG/cs.CL) on 2026-09-15; the two reports are consistent on all metrics.

• Autoregressive physics-grounded image-to-video model with structured scene memory: positional maps and object tracking maps derived online from previously generated frames
• Fine-grained motion control via sparse velocity-increment signals encoding physical quantities
• Two-stage training: bidirectional model finetuned with motion-control conditioning, then causal autoregressive model with structured scene memory
• Supports interactive mid-generation control of multi-object tabletop rigid-body scenes; one report calls it the first such method
• Reduces motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines
• Preferred by human evaluators in over 85% of in-the-wild comparisons
• Appeared on Hugging Face daily papers 2026-09-14 and arXiv cs.AI/cs.LG/cs.CL 2026-09-15