Private online learning and prediction for Littlestone classes
Differentially private online learning of Littlestone classes needs more mistakes than private prediction.
The paper separates differentially private online learning from private online prediction for every class of finite Littlestone dimension d under oblivious realizable adversaries. Any (ε, δ)-private online learner has a deterministic stream of length T whose expected mistakes are Ω((d/ε) log(T)^{2/3}). When δ is Θ(1/log T), private learning therefore needs Ω((log T)^{2/3}) expected mistakes, while private prediction admits an O((log log T)^2) bound. The sample-complexity gap grows with the time horizon.
- Online learning releases a hypothesis each round; prediction does not.
- Private learning needs Ω((d/ε) log(T)^{2/3}) expected mistakes.
- For fixed finite Littlestone dimension, prediction is O((log log T)^2).
- The learning-versus-prediction gap grows with the time horizon.
From the original article
1 min read at arxiv.orgWe study mistake bounds for differentially private online learning and online prediction under oblivious realisable adversaries. Online learning requires the learner to release a hypothesis at each time step whereas in online prediction, the learner only needs to make predictions without releasing a hypothesis.
By Amartya Sanyal · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.