RealMe builds 2K 4D-consistent volumetric video from sparse captures
Original title: RealMe: Immersive Volumetric Video from Minimal Captures
A high-resolution video diffusion model refines limited multi-view captures and beats prior methods on DNA-Rendering and RealVV.
RealMe is a framework that turns sparse, low-resolution captures into high-resolution (2K), 4D-consistent video for volumetric reconstruction. It encodes auxiliary capture data with layout-aware positional embeddings, adapts factorized spatio-temporal attention to the multi-view layout, and uses a latent frequency loss to close the gap with real captures. On the public DNA-Rendering benchmark and the authors' RealVV dataset, the paper reports higher visual fidelity and 4D consistency than prior methods.
- Output is described as 2K resolution and 4D-consistent.
- A latent frequency loss supervises generation in latent space.
- Evaluation uses DNA-Rendering and a self-captured RealVV dataset.
- The method targets sparse, low-resolution capture setups.
From the original article
1 min read at huggingface.co
This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources. Conventional dense multi-view reconstruction methods can produce high-quality volumetric videos, but their reliance on costly capture setups makes them impractical in real-world scenarios.
By Chenxu Zhou, Zhiyuan Yu, Yudong Jin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.