CausalAlign aims to keep camera ego-motion stable in video models
Original title: CausalAlign: Causal Alignment of Video Generation Models for Consistent Ego-Motion
Fine-tuned Wan2.2 gains more than 20% relative geometric consistency and lifts RoboCasa365 world-action success by 10 points.
CausalAlign targets camera ego-motion drift in diffusion video generators. Linear probes and causal interventions show a rise-and-fall pattern of ego-motion information during denoising. The method aligns selected layers to a 3D foundation model in delta-token space, which encodes frame-to-frame change rather than static scene content. With Wan2.2 fine-tuned on DL3DV and RealEstate10K, the authors report better motion accuracy, geometry, and temporal smoothness, plus higher success for world action models on RoboCasa365 mobile manipulation.
Unconfirmed: Improvement figures come from the authors' experiments only.
- Alignment uses delta-token space focused on motion dynamics.
- Wan2.2 is fine-tuned on DL3DV and RealEstate10K.
- Visual quality is reported as preserved.
- World action models gain 10 points on RoboCasa365.
From the original article
1 min read at huggingface.co
Diffusion-based video generation models (VGMs) can produce visually plausible videos, yet their camera ego-motion often drifts over long horizons. Camera ego-motion determines how the viewpoint changes between frames and is essential for maintaining consistent geometry and trajectories. To address this problem, we propose CausalAlign, a causal alignment framework with two components.
By Kai Zhu, Yintianrun Zhang, Shuwei Jiang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.