Plasticity Loss in Reinforcement Learning Studied in Stochastic Environments
Original title: Towards Understanding the Impact of Plasticity Loss on Reinforcement Learning in Stochastic Environments
Plasticity-preserving methods enhance robustness in noisy SAC tasks while stochasticity can regularize or accelerate plasticity loss.
The paper investigates plasticity loss effects on deep RL using SAC agents trained on three DeepMind Control tasks with controlled observation and action noise. Plasticity-preserving techniques increase learning stability on harder tasks, with advantages growing with noise levels. In easier environments noise acts as a regularizer sustaining plasticity; in harder ones it accelerates plasticity loss. Plasticity loss peaks in early training stages, with TD instability becoming dominant in the most challenging settings.
- Plasticity-preserving methods boost robustness
- Advantage grows with observation noise
- Stochasticity regularizes easy environments
- Accelerates plasticity loss in hard ones
- Most pronounced in early training
From the original article
1 min read at arxiv.orgDeep reinforcement learning (RL) is a powerful framework for sequential decision-making. Despite many high-profile successes in recent years, deep RL is known to be brittle. An emerging line of research attributes this brittleness to a loss of plasticity in neural networks.
By Philipp Bordne · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.