Reinforcement Learning with Conformal Action Sets: An Application to Sequential Recommendation
RLCP adapts sequential-recommendation action sets with calibrated pruning, lifting catalog diversity to 1.11–5.21× the best baselines.
The paper proposes Reinforcement Learning with Calibrated Pruning (RLCP), which adapts the retained action set in sequential recommendation using critic scores and an online threshold updated from binary proxy-target feedback. It proves a deterministic bound on the observed proxy miss rate and decomposes value loss into filtering and selection losses, yielding a finite session reward bound without requiring parameter convergence. On KuaiRand-Pure and MovieLens 1M, across 19 configurations, at least one RLCP variant achieved the highest catalog diversity, from 1.11× to 5.21× the strongest of four RL baselines, with competitive session depth and no larger retained sets.
- RLCP prunes the action set with critic scores and an online threshold.
- A value-loss decomposition separates filtering loss from selection loss.
- On 19 configurations, an RLCP variant led catalog diversity by 1.11× to 5.21×.
- Session depth stayed competitive without larger retained sets.
- Tags
- reinforcement-learningsequential-recommendationconformal-predictionaction-pruningmovielenskuairand
- Products
- KuaiRand-PureMovieLens 1M
From the original article
1 min read at arxiv.orgSequential recommenders typically use a fixed slate size even though the number of useful alternatives changes within a session. We propose Reinforcement Learning with Calibrated Pruning (RLCP), which adapts the retained action set using critic scores and an online threshold. The threshold is updated from binary feedback indicating whether the set contains an action in a proxy target.
By Wenwen Si · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.