UGG unifies video generation and 4D geometric prediction
Original title: UGG: Unified Generative Geometry Model
One model treats RGB, depth, and camera pose as conditional completion and can search scene hypotheses at inference.
Researchers introduce UGG, a unified generative model for 4D scene understanding that jointly models RGB video and geometric signals such as depth and camera pose. Different noise levels across modalities and time cast visual generation and geometric prediction as conditional completion in one model, so it can train on heterogeneous datasets with partial annotations. Inference-time search over scene hypotheses uses geometry inferred from observed RGB without ground-truth geometry. The authors report gains over non-unified baselines on camera-controlled video, depth, and pose, including sparse observations. Code and visualizations are at ugg-model.github.io.
Unconfirmed: Effectiveness claims are the authors' comparisons with non-unified baselines.
- UGG trains on datasets that lack some geometric or visual labels.
- Inference search uses geometric evidence from observed RGB frames.
- Evaluations cover video generation, depth estimation, and camera pose.
- Sparse observation settings are included in the reported tests.
From the original article
1 min read at arxiv.orgUnderstanding 4D scenes is a core challenge in visual intelligence, requiring models to jointly capture scene geometry, appearance, and dynamics from partial observations. However, existing approaches are subdivided into feedforward models for direct geometric prediction, and generative models for visual synthesis.
By Amani Kiruga · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.