Paper defines recommender regret for regularised RL estimates
Original title: Recommender regret for improving regularised RL estimation with bandits and MCTS
Bandit and MCTS estimators are recast as recommenders; experiments cover synthetic trees, arcade games, and scientific tasks.
The paper treats estimate generation in regularised reinforcement learning as a recommendation process, using bandit-based sampling as an improvement operator rather than scoring only sampling efficiency. It defines regulariser-aware recommender regrets, analyses lower bounds, and proves that simple algorithms such as uniform sampling can meet those bounds. Bandit- and search-based target improvement are unified under recommender Monte Carlo tree search. Across synthetic trees, arcade games, and scientific discovery tasks, the authors report that recommenders often converge faster and improve downstream performance, and they characterise regularised MCTS on scientific applications.
Unconfirmed: Faster convergence and better downstream performance are reported by the authors without numeric results in the abstract.
- The work targets Monte Carlo tree search and one-step look-ahead estimators.
- Uniform sampling is shown able to achieve the stated regret bounds.
- Experiments span synthetic trees, arcade games, and scientific discovery tasks.
- The paper says this is the first characterisation of regularised MCTS in scientific applications.
From the original article
1 min read at arxiv.orgRegularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide policy, value, or Q-value estimates in training and inference. However, existing bandit-based estimation frameworks evaluate sampling efficiency, not the quality of estimates.
By Danilo Vucetic · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.