VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis
VGGT-Diff routes VGGT-Ω geometry latents into a video diffusion model for sparse-view novel view synthesis.
VGGT-Diff is a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Visual tokens from VGGT-Ω carry 3D points and confidence, and a Visual Geometry Router turns them into query-aligned conditions for joint target-view denoising. Point-Track Residual Consistency and robust geometry conditioning improve multi-view stability. The authors report competitive or state-of-the-art interpolation and extrapolation across viewpoint difficulties.
- Routes VGGT-Ω visual geometry latents into a pretrained video diffusion model.
- Confidence-aware Visual Geometry Router keeps front and back surface evidence.
- Point-Track Residual Consistency regularizes residuals along reliable 3D tracks.
- Reports competitive or state-of-the-art interpolation and extrapolation results.
From the original article
1 min read at huggingface.co
We present VGGT-Diff, a geometry-routed multi-view diffusion model for sparse-view novel view synthesis. Existing novel view synthesis (NVS) methods face a fundamental trade-off: reconstruction-based approaches preserve observed geometry but struggle to synthesize unseen regions, while diffusion-based methods provide strong generative priors yet rely on implicit source-to-query correspondence.
By Kangjie Chen, Xiangyu Li, Dongbin Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.