From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
PARTS fine-tunes pretrained robot policies with RL on targeted bottleneck subtasks, boosting long-horizon manipulation success on YAM and Franka robots with minimal human intervention.
PARTS (Policy Adaptation with RL on Targeted Subtasks) is a real-world reinforcement learning framework that concentrates training on bottleneck subtasks where a frozen pretrained robot foundation policy repeatedly fails. On bimanual YAM tasks it raised complete-task success from 32% to 61%, and on single-arm Franka tasks from 50% to 95%, using tens of minutes of real-world RL rollouts per task. Agent-generated selectors and success verifiers activate residual corrections and provide local outcome rewards, enabling learning even when full-task successes are scarce. Under the same robot-rollout budget, PARTS exceeds existing real-world RL fine-tuning methods by more than 25% full-task success while requiring less human involvement.
- Subtask RL targets bottleneck steps where pretrained robot policies fail
- YAM bimanual success rose from 32% to 61%; Franka from 50% to 95%
- Requires only tens of minutes of real-world RL rollouts per task
- Beats existing real-world RL fine-tuning by over 25% under same rollout budget
From the original article
1 min read at huggingface.co
A pretrained robot foundation policy may execute most of a long-horizon task yet repeatedly fail at a few critical subtasks. Collecting additional full-task demonstrations for supervised fine-tuning (SFT) requires operators to repeat behaviors the policy already performs well.
By Sichang Su, Benjamin Yang, Zhiyun Deng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.