Offline RL paper regularizes policies via consequence distributions
Original title: Variational Consequence-Driven Offline Reinforcement Learning
Structural Behavior Regularization replaces an intractable dynamics constraint with a closed-form latent divergence and reports stronger offline RL benchmark results.
The paper argues that matching action distributions does not keep learned offline policies consistent with dataset transitions, because small action changes can produce very different next states. Structural Behavior Regularization (StBR) substitutes a latent divergence that the authors bound as a surrogate for consequence-distribution divergence. Regularizing with that divergence tightens the behavior constraint in dynamics-sensitive regions. The authors report superior results across diverse offline RL benchmarks.
- StBR targets dynamic consistency rather than pure action-distribution matching.
- The latent space is structured to reflect transition dynamics.
- The method tightens constraints where small deviations leave the offline dataset.
- No named datasets, model sizes, or numeric scores are given.
From the original article
1 min read at arxiv.orgOffline reinforcement learning (RL) typically mitigates distribution shift by imposing divergence constraints on the action distributions of the learned and behavior policies. However, this paradigm cannot ensure the underlying dynamic consistency structure: even minor action deviations can lead to drastically different transitions, inadvertently driving the agent away from the offline dataset.
By Wen Jiang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.