EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning
EgoLAP pretrains robot policies on human and robot data via language action reasoning, reaching 80.1% task progress.
EgoLAP is a vision-language-action pretraining framework that learns from egocentric human and robot trajectories through a shared language-based action chain-of-thought. It represents motion intent as structured, temporally abstracted language actions grounded in scene geometry, physics, and object affordances, aiming to bridge the embodiment gap. Across real-world and simulated experiments it reaches 80.1% mean real-world task progress, a 2.3x gain over alternative action representations. Motion-level reasoning also outperforms a composite format combining subtask, object-box, and visual-trace reasoning.
- Jointly trains on human and robot trajectories with language action reasoning.
- Motion intent is abstracted and grounded in geometry, physics, and affordances.
- Mean real-world task progress is 80.1%, a 2.3x gain over alternatives.
- Motion-level reasoning beats composite subtask, box, and visual-trace reasoning.
From the original article
1 min read at arxiv.orgEgocentric human data offer a path to scaling robot learning beyond costly robot demonstrations, yet the embodiment gap makes raw human trajectories a poor supervisory target for control. Our key insight is that, although low-level actions are embodiment-specific, their underlying motion intent can capture task-relevant structure that transfers across humans and robots.
By Lihan Zha · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.