HuRo: Robotizing Human Videos for Scalable VLA Pretraining
HuRo converts 630K human videos into robot-aligned episodes, improving VLA manipulation completion from 51.5% to 80.3% across four real tasks.
HuRo is a robotization pipeline that converts heterogeneous human videos into robot-aligned observations and action trajectories while inferring missing intermediate signals, yielding a dataset of about 630K episodes and 142M processed frames from five human-video sources. Pretraining vision-language-action models on this data improves real-world manipulation completion from 51.5% to 80.3% across four tasks, and OOD completion under spatial and visual shifts from 34.9% to 72.2%. Ablations show visual robotization improves OOD robustness and end-to-end pretraining with retargeted actions outperforms visual-only transfer.
- Pipeline robotizes 630K human-video episodes and 142M frames from five sources
- VLA pretraining lifts real-task completion 51.5% to 80.3%, OOD 34.9% to 72.2%
- End-to-end pretraining with retargeted actions beats visual-only transfer
From the original article
1 min read at huggingface.co
Human video datasets offer an abundant and diverse source of interaction data that can complement expensive real-robot data. To bridge the human-to-robot embodiment gap, existing approaches either robotize videos in task-matched settings or address observation and action alignment separately at scale.
By Jinho Jeong, Se June Joo, Jaehyun Kang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.