EyeRobot 2.0: Active Gaze for Precise Manipulation without Wrist Cameras
EyeRobot 2.0 achieves wrist-camera-level bimanual manipulation using only active gaze from a single stereo camera, outperforming passive stereo by 40% in real trials.
EyeRobot 2.0 uses Active Visual Fixation, physically swiveling two eye viewpoints onto a 3D fixation point and allocating more visual tokens to image centers. A hierarchical setup trains a low-level gaze servoing policy with dense geometric rewards, then a target selector co-trained with a behavior-cloning gripper policy; gripper information is canonicalized into a fixation-relative SE(3) frame. Across over 1,000 physical and 1,800 simulated trials, it outperforms passive stereo by 40% real and 20% sim, matches ego-plus-wrist policies (69% vs 64%), and more than doubles their success under wrist-camera occlusion (48% vs 22%).
- Active gaze with a single stereo camera replaces wrist cameras for fine manipulation
- Hierarchical RL: gaze servoing policy plus target selector co-trained with BC gripper policy
- Passive stereo alone drops real-world success from 52% to 27%; active gaze closes the gap
- Over 1,000 physical and 1,800 simulated trials across 13 tasks
- Organizations
- EyeRobot 2.0
From the original article
1 min read at arxiv.orgInspired by human vision, we introduce a framework using active gaze to enable fine-grained bimanual manipulation with only a single stereo camera. EyeRobot 2. 0 physically attends to a 3D fixation point in the scene by swiveling two eye viewpoints to center their gaze on it.
By Kush Hari · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.