When May a Bandit Leave Its Anchor? E-Process-Authorized Thompson Sampling under Non-stationarity
Paper proposes e-process-authorized Thompson sampling (e-ATS), using anytime-valid e-processes to decide when bandits may forget history under non-stationary rewards.
The paper introduces e-ATS, a Thompson sampling variant giving each arm full-history and discounted Beta states, where an anytime-valid e-process first authorizes the discounted state and a reversible relevance score controls its influence. Under a Beta-Bernoulli prior-predictive stationary model, e-ATS's probability of ever departing from optimistic Thompson sampling (OTS) is at most the chosen α_E without fitted thresholds. Removing authorization increased mean normalized dynamic pseudo-regret by 38.4% on the registered suite but reduced it by 7.5% on a literature-derived replay suite.
- Anytime-valid e-process authorizes when Thompson sampling may switch to discounted state
- Departure probability from optimistic Thompson sampling bounded by chosen alpha_E
- Removing authorization raised dynamic pseudo-regret 38.4% on registered suite
- On literature replay suite, removing authorization cut pseudo-regret by 7.5%
From the original article
1 min read at arxiv.orgStationarity rewards memory, but after a change the same history can mislead. We ask when forgetting should be permitted. E-process-authorized Thompson sampling (e-ATS) gives each arm full-history and discounted Beta states.
By Mayand Gulati · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.