Robust Policy Optimization via Adversarial Importance Sampling
Adversarial Importance Sampling estimates worst-case RL returns without extra interactions; authors also release the advrl PyTorch library.
The paper introduces Advis, which uses importance sampling over trajectories from standard training to estimate and optimize verifiable worst-case returns, requiring no additional environment interactions or auxiliary networks. It also releases advrl, a modular PyTorch library of single-file robustness methods and adversarial attacks for reproducible evaluation. The authors show adversarial hyperparameters do not transfer across agents, so they evaluate with 6-14x more attacker configurations than prior work. Effectiveness is demonstrated on continuous control environments.
- Advis estimates verifiable worst-case returns without extra environment interactions
- advrl library offers single-file implementations of robustness methods and attacks
- Attacker hyperparameters do not transfer across agents, risking robustness overestimation
- Evaluation uses 6-14x more attacker configurations than prior work
From the original article
1 min read at arxiv.orgSignificant progress has been made in safeguarding deep reinforcement learning (DRL) policies against input perturbations. Developing robust DRL involves three main stages: algorithm design, implementation, and evaluation. In this work, we identify and address a key limitation at each stage.
By Amine Andam · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.