ReLift recasts dense 3D tracking as trajectory repair
Original title: ReLift: Dense 3D Tracking as Lifted Trajectory Repair
ReLift repairs lifted trajectories from pretrained 2D and 3D models to achieve 2.1 APD gains on WorldTrack without retraining.
Dense 3D tracking from monocular video faces challenges with occlusions and joint learning of correspondences, depth and trajectories. ReLift lifts trajectories using strong pretrained components then applies a feed-forward corrector with alternating attention to repair corrupted positions and complete missing motion. Training on simulated errors with geometric inputs only allows the model to improve any tracker or geometry provider across datasets.
- Trains directly on 3D trajectories with simulated lifting errors
- Reuses model across trackers and geometry providers
- Improves strongest baselines by 2.1 APD on WorldTrack
- Faster configurations remain competitive with leading 3D trackers
- Organizations
- Hugging Face
- AI models
- ReLift
From the original article
1 min read at huggingface.co
Dense 3D tracking from monocular video requires establishing point correspondences over time, estimating their depth, and inferring their trajectories during occlusion. Existing methods typically learn these tasks jointly, a demanding objective given the scarcity of 4D supervision.
By Assaf Singer, Maroun Khriesh, Noam Rotstein · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.