Pixel-UMI pipeline for robot whole-body manipulation
Original title: Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
New method learns environment-aware whole-body policies from human demonstrations achieving 87% task progress.
Pixel-UMI combines Environment-Aware Retargeting and Retargeting-Error-Aware Masking to convert sensorized human trajectories into collision-aware robot actions despite embodiment differences. It uses DAgger corrections for contact-rich interactions and is evaluated on five manipulation tasks. The approach reduces viewpoint mismatches during deployment.
- 87.0% average task progress
- Handles articulated and deformable objects
- Uses DAgger for on-policy corrections
- Addresses kinematic and viewpoint errors
- Evaluated on five challenging tasks
From the original article
1 min read at arxiv.orgSensorized human demonstrations offer scalable data for mobile manipulation, but embodiment differences complicate policy learning. We present Pixel-UMI, a pipeline for learning environment-aware whole-body policies from human data.
By Jason Jingzhou Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.