4Director: Controlling Video World Models with Rigid 3D Geometry
4Director controls video world models using rigid meshes and per-frame transforms for consistent camera and object motion.
4Director conditions a video world model on an explicit 4D scene in which each object is reconstructed once as a canonical mesh and moved by one rigid transformation per frame. The controlled scene is rendered as a depth video, and a Motion Adapter synthesizes view-consistent appearance, illumination, and non-rigid dynamics. Training uses RealCOD-Rigid, a dataset of 20,774 clips with automatic rigid 3D annotations. Identity-Gated IoU jointly measures adherence to prescribed motion and preservation of object identity, and the method outperforms prior approaches on visual quality and camera and object control.
- Each object is one canonical mesh moved by rigid transforms.
- Motion Adapter converts a rendered depth video into final frames.
- RealCOD-Rigid provides 20,774 automatically annotated clips.
- IG-IoU scores prescribed motion and object identity together.
From the original article
1 min read at huggingface.co
Precise control over camera and object motion is essential for professional video production. Existing methods control objects only coarsely, through image-plane cues that are ambiguous in depth and rotation or through 3D tracks and blobs that lack complete geometry and lose consistency across viewpoint changes.
By Wei Cao, Hao Zhang, Vikram Voleti · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.