EgoXperience learns world-grounded egocentric video reconstruction
Original title: EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience
EgoX cuts camera, hand, and depth errors by 68.9%, 65.6%, and 42.9% versus the strongest baselines.
The paper presents EgoXperience, combining data collection, automatic annotation, a model, and a benchmark for joint egocentric reconstruction. EgoX-Engine couples geometrically verified object tracking with SLAM-based world alignment. EgoX, trained on EgoX-Data-6M and public annotations, reconstructs camera, scene, hands, and objects in one forward pass from monocular video and first-frame masks. Authors report state-of-the-art scene estimation and world-space hand-object pose recovery.
Unconfirmed: State-of-the-art and error reductions are the authors' own reported comparisons.
- EgoX-Data-6M is a 50-hour dataset with second-level task captions.
- EgoX-Bench is one hour selected across activity categories.
- Inputs are monocular egocentric video and first-frame object masks.
- Object pose scores improve by 28.0% against the strongest baselines.
- Products
- EgoX-Engine
- AI models
- EgoX
From the original article
1 min read at huggingface.co
Reconstructing human activity from egocentric video requires more than recovering individual 3D components: the observer's motion, surrounding geometry, and hand-object interactions must fit together in a consistent world. Achieving this coherence requires both joint reconstruction and aligned supervision, yet existing methods and datasets offer limited support for either.
By Xinhao Ji, Hui En Pang, Hao Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.