Agent as Policy for Robotic Manipulation
Agent as Policy lets a general-purpose agent drive a physical robot via runtime reasoning and program generation, reaching 100% success on manipulation tasks.
The paper introduces Agent as Policy (AGP), which puts task planning and execution for a physical robot under a general-purpose agent's control with no task-specific or environment-specific training. The agent interprets visual evidence, writes executable programs, issues motion commands, and revises actions based on physical outcomes. AGP was evaluated on real-world manipulation tasks including assembly from human videos, block construction from goal images, die reorientation, targeted throwing, and bimanual towel folding. It achieved success rates of 100%, 100%, and 80% on three block construction configurations.
- General-purpose agent directly controls a robot without task-specific training
- Agent writes programs, issues motion commands, and revises actions from visual feedback
- Achieves 100%, 100%, and 80% success on three block construction configurations
- Tasks span assembly, die reorientation, targeted throwing, and bimanual towel folding
From the original article
1 min read at huggingface.co
We demonstrate that a general-purpose agent can directly drive a physical robot throughout task execution without any task-specific or environment-specific training. We introduce Agent as Policy (AGP), which places task planning and execution under the agent's control.
By Mengzhao Jia, Yang Lin, Xixin Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.