A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control
Success Guided Sampling focuses mega-scale robot RL on the policy frontier, solving tasks prior methods miss.
The paper introduces Success Guided Sampling, an adaptive sampler for sim-to-real reinforcement learning that focuses training on task configurations near the frontier of a policy's current ability. Uniform sampling of diverse simulator resets wastes experience on setups the policy has already mastered or cannot yet attempt. Using up to 2^20, more than one million, parallel environments, SGS solves multi-terrain quadruped locomotion and contact-rich assembly tasks that prior methods fail. The authors distill manipulation policies into RGB policies and demonstrate zero-shot transfer to several assembly tasks on real hardware.
- SGS concentrates training near the current capability frontier
- Runs use up to 2^20, over one million, parallel environments
- Solves multi-terrain quadruped and contact-rich assembly tasks
- Distilled RGB policies transfer zero-shot to real hardware
From the original article
1 min read at arxiv.orgGeneral-purpose robots must perform a wide range of tasks from agile locomotion to dexterous manipulation. While sim-to-real reinforcement learning (RL) has proven to be a useful tool for this goal, current RL pipelines depend on engineering-heavy, per-task structural priors such as shaped rewards and demonstrations.
By Octi Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.