MOMENT-DP ranks private noise mechanisms using optimizer momentum
Original title: MOMENT-DP: Momentum Aware Evaluation of Correlated Noise for Differentially Private Training
Holding curvature, schedule and privacy calibration fixed, changing only momentum can reverse mechanism rankings.
The paper introduces MOMENT-DP, a framework that models how temporally correlated privacy noise moves through momentum-based optimization. For locally quadratic objectives it expresses privacy-induced final loss with a transfer operator covering curvature, learning-rate scheduling, momentum and noise covariance, recovering NoiseCurve’s criterion as the zero-momentum, constant-learning-rate case. Changing only momentum, with candidate mechanisms fixed, produces genuine ranking reversals. The effect also appears in per-example-clipped private logistic regression but remains unresolved in the neural regime studied.
Unconfirmed: Whether momentum changes mechanism rankings for neural training is left unresolved by the authors.
- NoiseCurve’s quadratic criterion is the zero-momentum, constant-rate special case.
- Ranking reversals occur with curvature, schedule and privacy calibration held fixed.
- The effect persists in clipped private logistic regression.
- Results in the studied neural regime remain unresolved.
From the original article
1 min read at arxiv.orgTemporally correlated noise can substantially improve the utility of differentially private optimization, yet candidate mechanisms are commonly designed or evaluated using only partial information about the optimization process, such as workload structure, learning-rate schedules, or local curvature.
By Sunny Gupta · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.