Oracle-Efficient and Parameter-Free Agnostic Smoothed Online Learning
A parameter-free Follow-The-Perturbed-Leader method gets near-optimal agnostic smoothed online regret without knowing the base measure.
Smoothed online learning assumes each covariate's conditional density is at most 1/σ relative to a fixed base measure, interpolating between adversarial and stochastic data. Prior oracle-efficient algorithms required sampling access to that measure or labels perfectly predicted by a fixed hypothesis. The authors give the first oracle-efficient algorithm for the agnostic setting without knowing the base measure, using Gaussian Follow-The-Perturbed-Leader. It needs no knowledge of the measure, smoothing parameter, or horizon, and achieves regret Õ(d√(T/σ)) for binary classes of VC dimension d with one empirical-risk-minimization call per round, optimal up to a √d factor.
- Removes the need to know the base measure or smoothing parameter.
- Gaussian Follow-The-Perturbed-Leader is parameter-free and oracle-efficient.
- Regret is Õ(d√(T/σ)) with one ERM oracle call per round.
- The bound is optimal up to a square-root-of-d factor for VC classes.
From the original article
1 min read at arxiv.orgOnline learning is an attractive framework in many domains because it permits well-defined learning even when data are dependent or chosen adversarially. This generality, however, comes at a steep price, introducing significant statistical and computational barriers.
By Sasha Voitovych · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.