Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
Theory paper proves fast finite-sample rates for dually regularized adversarial imitation learning.
The paper studies adversarial imitation learning, where a policy imitates experts by competing against a reward that distinguishes expert and learner behavior. Its model-free Dually Regularized AIL algorithm combines KL policy regularization with a quadratic reward penalty weighted by expert and learner occupancies. With K online episodes and N expert trajectories, it proves a tilde-O(1/K + 1/N) bound on the regularized imitation gap. The authors say it is the first method to achieve simultaneous tilde-O(1/epsilon) sample complexity in both expert demonstrations and online interactions for this objective, including stochastic experts.
- Dually Regularized AIL combines KL policy regularization with a quadratic reward penalty.
- Proves a tilde-O(1/K + 1/N) bound on the regularized imitation gap.
- Claims simultaneous tilde-O(1/epsilon) rates for demonstrations and online interaction.
- Analysis covers finite-horizon MDPs with general function approximation.
From the original article
1 min read at arxiv.orgWe study adversarial imitation learning (AIL), in which an agent learns to imitate expert demonstrations by optimizing a policy against an adversarial reward that distinguishes expert and learner behavior. Historically, reward regularization and entropy-based policy regularization are key components of empirically successful methods such as GAIL and LS-IQ, yet their finite-sample benefits remain underexplored.
By Hanbin Zhou · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.