Groupoid-Based Internal State Representations for Reinforcement Learning with Local Symmetries
Groupoid-based RL discovers local, state-dependent symmetries during interaction, learning in a symmetry-reduced space and beating standard Q-learning efficiency.
The paper proposes a reinforcement learning framework using groupoids to capture local, state-dependent symmetries and dynamically discover equivalence structures during interaction. The agent maintains orbit representatives together with transporters that map raw states to canonical forms, enabling learning and decision-making in a symmetry-reduced space while preserving local distinctions. Empirical results show improved sample efficiency and convergence over standard Q-learning in dense and large-scale environments with strong partial symmetries.
- Groupoids capture local, state-dependent symmetries beyond fixed group actions
- Orbit representatives and transporters map raw states to canonical forms
- Learning occurs in symmetry-reduced space preserving local distinctions
- Improves sample efficiency over standard Q-learning in partially symmetric environments
From the original article
1 min read at arxiv.orgSymmetries play a central role in reducing the complexity of reinforcement learning problems, yet most existing approaches rely on fixed group actions or predefined state abstractions.
By Ben Opperman · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.