Linear Bandits under Exact Sliding-Window Constraints
A rare-switching bandit algorithm keeps exact sliding-window feasibility with near-optimal regret.
The paper studies linear bandits where every consecutive window of actions must lie in a prescribed feasible set. Offline, convexity and cyclic-shift invariance make a stationary solution optimal when the window divides the horizon, and within an additive O(w) gap otherwise. Online, geometric structure alone does not guarantee learning; the authors define a transition diameter and a rare-switching OFUL algorithm with regret on the order of d times square-root of T plus terms in the diameter and window. For general non-stationary constraints they obtain a similar bound using a history-state diameter, and report exact feasibility with fewer policy updates on real and synthetic benchmarks.
- Every consecutive block of actions must stay inside a prescribed feasible set.
- Stationary solutions are optimal offline when the window divides the horizon.
- A transition diameter captures when sublinear regret is possible online.
- Rare-switching OFUL achieves regret scaling with dimension, diameter, and window size.
From the original article
1 min read at arxiv.orgWe study linear bandits under exact sliding-window constraints, where every consecutive block of actions must belong to a prescribed feasible set. In the offline setting, where the reward function is known, we show that convexity and cyclic-shift invariance make a stationary solution optimal when $w\mid T$ and within an additive $O(w)$ gap otherwise.
By Seyed Mohammad Hadi Hosseini · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.