Multiscale FFT algorithm improves privacy loss accounting for random allocation
Original title: Fast Privacy Loss Accounting for Random Allocation via Multiscale FFT
Multiscale FFT reduces runtime from quadratic to O(α^{-1} log(1/α)) and rounding errors to O(α^2/log(t)) for DP-SGD with PREAMBLE.
The paper introduces a multiscale FFT method to compute exact privacy loss distributions for random allocation sampling, where each user's data is used in k randomly chosen steps from t total steps. It represents likelihood ratios on dyadic grids and uses FFTs per interval to achieve the improved time bound while preserving guarantees. Mean-preserving connect-the-dots projections further reduce errors in mean and χ² divergence. Experiments demonstrate speedups of several orders of magnitude at high accuracy.
- Runs in O(α^{-1} log(1/α)) time vs prior O(α^{-2})
- Improves rounding errors to O(α^2 / log(t))
- Applies to private optimization and communication-efficient aggregation
- Demonstrates speedups for DP-SGD with PREAMBLE
- Exact pairwise sum computation via FFTs
From the original article
1 min read at arxiv.orgIn the random allocation sampling scheme each user's data is used in $k$ steps chosen randomly and uniformly from a sequence of $t$ steps. This scheme is known to improve on Poisson sampling in private optimization and in communication-efficient aggregation.
By Vitaly Feldman · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.