HuGe pretrains humanoid VLAs on 2,000 hours of egocentric data
Original title: HuGe: Scaling Humanoid Generalist Vision-Language-Action Models with Egocentric Experience
The three-stage pipeline reports state-of-the-art simulation results and higher Unitree G1 pick-and-drop success.
HuGe is a family of humanoid generalist vision-language-action models pretrained on more than 2,000 hours of egocentric human and robot data. It predicts latent motion tokens for a pretrained whole-body controller plus hand-control signals, using egocentric pretraining, humanoid adaptation, and downstream fine-tuning. Transfer uses simulated pseudo-proprioception and re-rendered arms. The authors report state-of-the-art results on HumanoidArena and higher real-world pick-and-drop success on a Unitree G1 when robot demos are mixed with adapted egocentric video.
- Pretraining uses over 2,000 hours of egocentric human and robot data.
- Three existing VLA architectures predict latent motion tokens and hand signals.
- Simulation playback builds pseudo-proprioceptive states during pretraining.
- Real tests are on the Unitree G1 pick-and-drop task.
- Vendors
- Unitree
- Products
- Unitree G1
- AI models
- HuGe
From the original article
1 min read at huggingface.co
Extending Vision-Language-Action models (VLAs) to humanoid robots requires coordinated whole-body control and diverse demonstrations that are costly to collect in real-world scenarios. Although egocentric human demonstrations provide a complementary source of task experience, differences in visual embodiment and missing robot proprioception complicate transfer to humanoid policies.
By Yang Liu, Jiacheng Liu, Hengtao Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.