World Action Learning via Interaction-Centric Spectral Latent Guidance
WING transfers interaction knowledge from egocentric human video to robot policies via spectral latent guidance.
WING transfers manipulation knowledge from human egocentric video to robot policies by isolating interaction-centric latent actions from observer motion. Shared slowly varying spectral components between human latent actions and robot behavior guide action generation across embodiments. Reported average success is 99.20% on LIBERO, 93.80% on RoboTwin 2.0, and 57.7% on RoboCasa-GR1, with strong results on four real-world tasks.
- Separates ego-camera motion from hand-object interaction in egocentric video.
- Uses shared low-frequency spectral structure to guide robot actions.
- Reports 99.20% LIBERO and 93.80% RoboTwin 2.0 success.
- Also scores 57.7% on RoboCasa-GR1 and four real manipulation tasks.
From the original article
1 min read at huggingface.co
Learning general-purpose robot policies requires large-scale real-world interaction data, yet collecting robot demonstrations remains expensive and difficult to scale.
By Zhiming Liu, Yikun Miao, Ying Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.