GSBR raises robotic RL success from 33% to 95%
Original title: GSBR: Geometric Support Behavior Regularization for Online RL
Geometric Support Behavior Regularization allows base-policy improvements while penalizing actions outside support on 19 robotic tasks.
Base policies from imitation learning are too restrictive when used for RL regularization. GSBR penalizes actions by distance from base-policy support estimated from finite samples. Actions inside support incur no penalty, enabling concentration on high-value modes. Across 19 tasks on four robotic benchmarks, GSBR lifts average success rate from 33% to 95% and achieves highest throughput among compared methods.
- Penalizes actions outside base policy support
- Requires only sample access to base policy
- Success rate rises from 33% to 95%
- 23% higher normalized throughput than next-best RL method
- GSBR estimates distance via finite base-policy samples
From the original article
1 min read at arxiv.orgRegularizing online reinforcement learning (RL) toward a pretrained base policy can improve sample efficiency and learning stability, but regularization that is too restrictive limits how much the policy can improve. Base policies -- typically diffusion or flow-matching models trained by imitation learning on large, diverse datasets -- capture many behaviors that are reasonable but suboptimal.
By Charles Xu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.