Paper derives optimal privacy-budget schedules for private SGD
Original title: When to spend privacy: optimal allocation in private learning
PB-AMD uses adjoint gradients and entropic updates so noise schedules stay positive and spend the certified zCDP budget.
The paper treats the full Gaussian noise-variance schedule in differentially private SGD as an open-loop control problem under a composed zero-concentrated differential privacy constraint. For expected quadratic terminal tracking, it proves every local minimizer exhausts the certified budget and reduces the problem to an exact-budget simplex. Privacy-Budget Adjoint Mirror Descent combines pathwise adjoint gradients with entropic updates that keep positivity, charge floors, and the prescribed budget. A linear-quadratic case yields an explicit square-root allocation law. Experiments report better tracking than several rules and better prediction than uniform allocation when learning rates restart, with lower schedule-optimization time than projected gradient descent at a shared stationarity tolerance.
- The constraint is composed zero-concentrated differential privacy.
- A linear-quadratic reduction gives an explicit square-root allocation law.
- Learning-rate allocation performed similarly in the restart setting.
- Guarantees cover existence, budget stability, and several stationarity notions.
From the original article
1 min read at arxiv.orgPrivacy accounting bounds the cumulative privacy cost of Differentially Private Stochastic Gradient Descent but does not determine how to allocate a fixed budget across iterations. We formulate the complete Gaussian noise-variance schedule as an open-loop control under a composed zero-Concentrated Differential Privacy constraint, explicitly accounting for schedule selection.
By Umberto Biccari · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.