Better Batches, Better Privacy: Random Allocation for DP-BandMF
Random-allocation batching for DP-BandMF cuts RMSE up to 1.85 times and speeds privacy accounting on GPUs.
The paper improves differentially private training that combines DP-BandMF with privacy amplification by sampling. New batch-selection strategies inspired by random allocation reduce RMSE by up to 1.85 times versus prior Poisson-inspired DP-BandMF methods. An optimized GPU accountant raises sampling and scoring throughput by about two orders of magnitude. Jointly choosing the strategy matrix and batch policy trims RMSE by up to 13.4 percent versus fixed matrices.
- Random allocation beats Poisson-style batches for DP-BandMF.
- RMSE improves by up to 1.85 times.
- GPU accounting throughput rises by roughly 100 times.
- Joint matrix and batch selection cuts RMSE up to 13.4 percent.
From the original article
1 min read at arxiv.orgThe current state of the art for batch selection and noise addition in differentially private training is a combination of DP-BandMF (informally, DP-SGD where the noise added in two different steps can be correlated if the steps are within $b$ of each other) and privacy amplification by sampling via Monte Carlo accounting.
By Ryan McKenna · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.