HumanoidPose estimates robot joint pose from video alone
Original title: Vision as Proprioception: Estimating Humanoid Robot Pose from Video
On three real humanoid platforms, joint-angle error fell 18-32% versus the best baseline on each.
HumanoidPose recovers full-body joint configurations in world space from video when onboard robot state is unavailable. It combines pretrained human-pose features with robot kinematics and learns from human motions retargeted to sixteen humanoid embodiments. Unlabeled online videos are used via self-training to train a 2D robot-pose detector. The authors report better joint-angle and joint-position accuracy than evaluated baselines, plus improved global localization and generalization to unseen embodiments.
- Training uses synthetic data from sixteen retargeted humanoid embodiments.
- Self-training on unlabeled online videos builds a 2D robot-pose detector.
- Joint-angle error dropped 18-32% versus the best baseline on each of three platforms.
- The method also reports better global localization and unseen-embodiment generalization.
From the original article
1 min read at huggingface.co
Estimating a humanoid robot's internal state from video has a wide range of real-world applications but remains unexplored. During interactions with a robot, its onboard state information may be unavailable due to privacy, ownership, or communication constraints.
By Jiaxi Jiang, Matthias Heyrman, René Zurbrügg · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.