Private expert prediction reaches square-root-of-best-loss regret
Original title: Private Online Prediction from Experts with Small Losses
For constant privacy and oblivious inputs, the bound matches the non-private lower bound up to polylog factors.
A new differentially private online-prediction-from-experts algorithm achieves regret on the order of the square root of the best expert’s cumulative loss, improving on prior private bounds of square root of the time horizon when that loss is much smaller. The guarantee assumes the input sequence is fixed in advance and independent of the algorithm’s outputs. The authors also show that adaptive inputs force square-root-of-horizon regret even when the best expert’s loss is zero.
- Prior private algorithms only reach near-optimal regret scaling with the square root of T.
- Privacy adds only polylog factors versus the non-private lower bound for constant privacy.
- Adaptive sequences still require square-root-of-T regret even if the best loss is zero.
- The positive result needs an oblivious input sequence fixed before the algorithm runs.
From the original article
1 min read at arxiv.orgWe consider online prediction from experts, a fundamental problem in machine learning, under differential privacy constraints. Existing private algorithms achieve near-optimal regret $\tilde{O}(\sqrt{T})$, where $T$ is the time horizon. However, this bound becomes suboptimal when $L^\star$, the cumulative loss of the best expert, is significantly less than $T$.
By Bo Li · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.