LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
LEGO turns one exocentric video into an egocentric view with a lifting-free synthesizer and diffusion model.
LEGO generates an egocentric video from one exocentric recording without depth estimation, point clouds, or reprojection. An LVSM-style transformer renders the target view directly and supplies structural conditions plus per-region confidence to a video diffusion model that restores fine detail. The authors report consistent gains over the explicit lifting pipeline and generalization to other datasets without retraining.
- Generates egocentric video from a single exocentric recording.
- Replaces depth lifting and point-cloud reprojection with a learned view synthesizer.
- Per-region confidence masks weak areas and guides early denoising.
- Outperforms the explicit pipeline and generalizes to other datasets without retraining.
From the original article
1 min read at huggingface.co
Generating an egocentric video from a single exocentric recording is a challenging case of novel view synthesis, as the two cameras share little overlap and much of the target view is unobserved. Current state-of-the-art methods reconstruct the scene explicitly by estimating depth, lifting the video into a point cloud, and re-rendering it from the egocentric camera to condition a video diffusion model.
By Suhwan Cho, Yonwoo Choi, Soongjin Kim · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.