Co-Evolving Robot Orchestrators and Policies through Deployment
Robo-COP co-evolves robot policies and orchestrators in deployment, raising real-world success from 38.3% to 50%.
Robo-COP co-evolves a vision-language orchestrator and a vision-language-action policy during robot deployment instead of freezing the policy. It curates skill demonstrations from its own executions, fine-tunes the policy when data can fix recurring failures, and accepts a new policy only after verified skill gains. Across ten simulated RoboLab tasks, mean held-out success rose from 64.8% to 73.8%, versus 65.8% for unverified fixed-schedule fine-tuning. On three real-world tasks, held-out success rose from 38.3% to 50.0%.
- Robo-COP jointly updates a VLM orchestrator and a VLA policy from deployment data.
- New policies are adopted only after they improve the skills they were trained for.
- Simulated held-out success rose from 64.8% to 73.8% across ten RoboLab tasks.
- On three real-world tasks, success rose from 38.3% to 50.0%.
From the original article
1 min read at huggingface.co
Vision-language-action (VLA) policies trained on large datasets are capable within their training domains, yet they still fail to generalize to the variety of situations a robot meets in real-world deployment. Agentic robot systems complement the policy with a vision-language model (VLM) orchestrator that learns when to call the policy, how to instruct it, and when to use scripted skills instead.
By Xilun Zhang, Maggie Wang, Erik Bauer · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.