HiP3R aligns long video 3D reconstructions with hierarchical pose graphs
Original title: HiP3R: Hierarchical Pose Graphs for Long-Horizon 3D Reconstruction
A chunk-level Sim(3) graph sets global scale, then one shared SE(3) camera is estimated per frame, with no extra training.
Feed-forward models estimate camera motion and 3D geometry only on short windows, so long videos are split into overlapping chunks with inconsistent frames and duplicate cameras. HiP3R first optimizes a chunk-level Sim(3) pose graph for global layout and scale, then holds that layout fixed and estimates one shared SE(3) camera per frame. The method needs no additional training or fine-tuning and works with chunk-based and streaming backends. Experiments on multiple benchmarks show improved camera trajectories and 3D reconstruction.
- Direct inference is blocked on sequences of thousands of frames by cost.
- Chunk-level graphs close large loops but cannot fix internal frame errors.
- Frame-level optimization is flexible but expensive and can break metric scale.
- Code is available at kkakkkka.github.io/HiP3R.
From the original article
1 min read at huggingface.co
Feed-forward models accurately estimate camera motion and 3D geometry from short video windows. However, computational and memory costs prevent direct inference on sequences containing thousands of frames. Long videos must therefore be processed as overlapping chunks.
By Zunnan Xu, Shangchen Zhou, Yihang Luo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.