Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration
Dex-One2Many learns a generalizable dexterous manipulation policy from one human video via scene-graph-guided reinforcement learning.
Dex-One2Many is a real-to-sim-to-real framework that turns a single human demonstration video into sequential scene graphs to guide reinforcement learning for dexterous manipulation. The graphs constrain object relations rather than exact poses, sampling diverse reset states and giving dense stage rewards so exploration stays short while covering unseen poses and grasps. Trained only in simulation, the policy transfers zero-shot to a real multi-fingered hand. Across five tool-use and manipulation tasks it exceeds baselines by 6.5% on seen configurations and by 71% on unseen scenarios.
- Abstracts one human video into sequential scene graphs that guide RL
- Graphs sample diverse resets and supply dense per-stage rewards
- Policy transfers zero-shot from simulation to a real multi-fingered hand
- Beats baselines by 6.5% seen and 71% unseen across five tasks
From the original article
1 min read at huggingface.co
While learning dexterous manipulation from a single human video offers a promising alternative to costly robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the video.
By Jusuk Lee, Sungha Kim, Yeonsoo Park · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.