Fréchet Forcing stabilizes long world-model rollouts from short clips
Original title: Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips
MiniWorld-1B, fine-tuned for at most 36 GPU-hours on 381-frame clips, stayed stable past 4,000 frames.
Fréchet Forcing fine-tunes an action-conditioned streaming world model and its multi-step sampler with three constraints drawn from short real clips: generative anchoring, a Fréchet distance for scene fidelity, and a Fréchet distance to real frames for realism. Training needs no teacher model, online critic, or long videos. Fine-tuning MiniWorld-1B, pretrained on 253-frame clips, for at most 36 GPU-hours on 381-frame clips produced rollouts stable beyond 4,000 frames that still followed actions in robot manipulation and camera-controlled exploration.
Unconfirmed: Stability and action-following claims are the authors' reported measurements.
- MiniWorld-1B was pretrained on 253-frame clips.
- Fine-tuning used 381-frame clips and at most 36 GPU-hours.
- Reported rollouts remained stable beyond 4,000 frames.
- Action following held in robot manipulation and camera-controlled exploration.
From the original article
1 min read at huggingface.co
Streaming video world models generate the future chunk by chunk, conditioned on their own past and a stream of actions. Trained on short clips, they drift once a rollout exceeds the training horizon, as prediction errors re-enter the context and compound. A common remedy is to train the model on its own rollouts via distribution matching distillation (DMD).
By Yian Zhao, Yu Yan, Hongcan Guo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.