Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model
Latent-flow world models lose object motion; decode-augmented training restores it without unfreezing the representation.
The paper finds that latent world models trained in a frozen self-supervised space lose the motion needed for object manipulation. Anchor-sparse, latent-only supervision fails to place change along the horizon, and retraining with those losses yields teleport-like motion. Decode-augmented rollout training keeps the representation frozen and retrains only the flow, restoring temporal structure and scene coupling. At larger scale it closes nearly half the remaining gap to an oracle interpolation reference, and the authors warn that pixel error rewards frozen predictions.
- Frozen latent-flow world models silently lose object motion.
- Latent-only retraining produces teleport-like rather than continuous motion.
- DART retrains only the flow using decode-path supervision.
- Pixel error alone can reward predictions that stay frozen.
Full article143 words · extracted from arxiv.org · click to collapse
Latent world models that integrate a flow in a frozen self supervised latent space train stably and cheaply, yet silently lose the property manipulation depends on most: motion. The pretrained flow never moves the manipulated object; retraining it with latent-only losses only trades stillness for teleport-like motion. We trace the failure to the training signal, not the representation: anchor-sparse, latent-only supervision never says where along the horizon change belongs. Decode-augmented rollout training (DART) repairs this while keeping the representation frozen, retraining only the flow with decode-path supervision. DART outperforms its latent only parent on the full protocol, restores the temporal structure of motion, and re-couples predicted motion to the scene; at larger scale it further improves prediction quality, closing nearly half the remaining gap to an oracle-informed interpolation reference. Finally, we report an unexpected finding about evaluation: pixel error alone rewards frozen predictions.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.28414