OmniShowcase generates subject-consistent video from multi-view images
Original title: OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images
Training covers 12 camera motions and 88 trajectories, including full 360-degree orbits, with surface-normal conditioning from reference views.
OmniShowcase addresses subject-to-video generation when the camera reveals surfaces absent from a few reference images. An automated pipeline renders training videos from subject geometry, and the model is conditioned on surface-normal maps that need not align temporally with generated frames. Showcase-4K provides those videos; ShowcaseBench is a held-out set with source meshes and camera parameters. The authors report state-of-the-art subject identity and 3D consistency with competitive video quality and motion.
Unconfirmed: State-of-the-art identity and consistency claims are the authors' evaluation results.
- Conditioning uses a turntable of reconstruction-derived surface-normal maps.
- Viewpoint changes in training are driven by underlying subject geometry.
- ShowcaseBench keeps the meshes and camera parameters used to render each video.
From the original article
1 min read at huggingface.co
Generating videos of a specific subject from a few reference images remains challenging when the camera reveals surfaces that are not visible in the input. Early subject-to-video (S2V) methods condition on a single reference image and must therefore infer the subject's unseen appearance and geometry as the viewpoint changes.
By Junyeong Ahn, Geonwoo Kim, Kinam Kim · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.