TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations
TrackEverything densely tracks all visible 3D points beyond 1000 frames within 40 GB, beating dense trackers by over 20% APD.
TrackEverything is a dense 3D point tracker that stores videos as persistent world-coordinate scene tracks so complexity scales with unique geometry rather than duration. It de-duplicates co-located tracks with voxelization, splits endpoint prediction and static-dynamic classification from a lightweight dynamic trajectory refiner, and uses 3D WAFT feature sampling instead of 4D correlation volumes. The authors report tracking all visible points in videos longer than 1000 frames within 40 GB of GPU memory, with more than 20% higher APD than open-source all-frame dense 3D trackers on short TAPVid-3D clips while staying competitive with sparse long-sequence trackers.