A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning
Paper unifies regularization-based robust RL methods via new performance-gap upper bounds and jointly learned Lagrange multipliers.
The authors derive new upper bounds on the gap between nominal and worst-case deep RL policies, each expressible as an existing regularization objective plus a KL-divergence penalty. Robust training is reformulated as constrained optimization, where prior methods correspond to a fixed Lagrange multiplier. The multiplier is instead updated jointly with the policy, auto-tuning the regularization weight. Adversarial evaluations across several continuous control tasks validate the theory.
- New upper bounds link regularization objectives to worst-case performance gaps
- KL penalty between nominal and worst-case policies explains robustness gains
- Lagrange multiplier learned jointly with policy for automatic weight tuning
- Validated with adversarial evaluations on continuous control tasks
From the original article
1 min read at arxiv.orgRegularization-based methods have become a standard approach for training Deep Reinforcement Learning policies against adversarial input perturbations. In this paper, we unify these methods by deriving new upper bounds on the performance gap between the nominal and worst-case policies.
By Amine Andam · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.