TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations
TrackEverything tracks all visible 3D points in videos beyond 1,000 frames within 40 GB by de-duplicating scene geometry.
TrackEverything represents videos as persistent 3D scene tracks in world coordinates so cost scales with unique geometry rather than duration. It de-duplicates co-located tracks at sliding-window boundaries, classifies static versus dynamic points, and samples features in the scene cloud with 3D WAFT instead of 4D correlation volumes. The authors report tracking all visible points in videos longer than 1,000 frames within 40 GB of GPU memory. On TAPVid-3D it outperforms open-source all-frame dense 3D trackers by more than 20% APD on short clips.
- Decouples tracking cost from video length using persistent 3D scene tracks.
- Voxel de-duplication merges repeated observations of the same surface.
- Tracks all visible points past 1,000 frames within 40 GB GPU memory.
- Beats open-source dense 3D trackers by over 20% APD on short clips.
- Products
- TAPVid-3D
- AI models
- TrackEverything
From the original article
1 min read at arxiv.orgExisting point tracking models face a fundamental tradeoff: they can either track a sparse set of query points over long horizons, or track all points across only short clips. We introduce TrackEverything, a 3D point tracker that breaks this trade-off by representing videos as persistent 3D scene tracks in world coordinates.
By Ayush Jain · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.