Video-Conditioned Generative Joint 2D-3D Hand Motion Recovery
JoHan recovers smoother 2D and 3D hand motion directly from video without per-frame pose estimates.
JoHan is a generative framework that recovers joint 2D and 3D hand motion directly from video without intermediate per-frame pose predictions. Trained from scratch, it learns temporal dynamics and cross-representation correspondence so generated 2D trajectories guide 3D reconstruction and recovery of global position and orientation relative to the camera. Experiments on challenging benchmarks report higher accuracy and speed for local hand pose and camera-space reconstruction, with smoother motion than prior methods while keeping high per-frame pose accuracy.
- JoHan generates aligned 2D and 3D hand pose sequences from video.
- It avoids unreliable intermediate per-frame pose estimates.
- 2D trajectories guide 3D reconstruction and camera-relative pose.
- Authors report smoother motion with high per-frame accuracy and faster inference.
From the original article
1 min read at huggingface.co
Recovering faithful 3D hand motion from video remains challenging due to frequent occlusions and incomplete visual observations, which make frame-wise pose estimates unreliable and temporally inconsistent. To address this problem, we propose JoHan, a unified generative framework that recovers hand motion directly from video sequences without relying on intermediate per-frame pose predictions.
By Chen Xu, Yunqi Li, Binbin Huang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.