Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
A closed-form noise formula bounds membership-inference accuracy for random-allocation DP-SGD and needs less noise.
For random-allocation DP-SGD, where each record is used once per epoch at a random step, the authors give a closed-form cap on membership-inference accuracy of one half plus one quarter times the square root of (1+(e^(1/sigma^2)-1)/M)^E minus 1, assuming equal priors. Derived from chi-square divergence between a Gaussian and a dominating mixture, it returns sigma in about a microsecond and needs at most about half the noise of the prior closed-form bound. An exact expression for those distributions needs 13.0% to 20.2% less noise, matches a published accountant in test accuracy, and empirical attacks on trained models stayed below the bound.
- Closed-form bound limits MIA accuracy using only M, E, and sigma.
- It needs at most about half the prior closed-form noise.
- Exact calibration uses 13.0% to 20.2% less noise than the formula.
- Trained-model membership inference stayed below the analytic bound.
From the original article
1 min read at arxiv.orgDP-SGD protects training data by adding Gaussian noise to clipped gradients. The amount of noise is usually chosen by running a numerical privacy accountant inside a search. We study DP-SGD with random allocation, where each epoch uses every record once, at a randomly chosen step.
By Murat Bilgehan Ertan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.