Counting on Dummies enables privacy amplification with truncated batches
Mechanism uses dummy points to privatize batch sizes in DP-SGD while preserving privacy under truncation and subsampling.
Stories per day · 30 days
3 RSS
Relateddifferential-privacy5privacy-accounting2dummy-points1fft1hyperparameter-transfer1medical-records1membership-inference1model-training1noise-schedule1optimization1privacy-amplification1random-allocation1synthetic-data1truncated-batches1
9 stories, newest first · one card per story
Stories tagged #dp-sgd
Multiscale FFT algorithm improves privacy loss accounting for random allocation
Multiscale FFT reduces runtime from quadratic to O(α^{-1} log(1/α)) and rounding errors to O(α^2/log(t)) for DP-SGD with PREAMBLE.
Paper derives optimal privacy-budget schedules for private SGD
PB-AMD uses adjoint gradients and entropic updates so noise schedules stay positive and spend the certified zCDP budget.
DP-μP transfers learning rates under differential privacy
Scaling rules cover DP-SGD, DP-Adam, and DP-Muon on MLP, ViT, and GPT-2.
Study compares privacy and capability of synthetic medical-record training
Direct finetuning leaked most; DP-SGD cut leakage but hurt capability; privacy-aware rewriting balanced both better.
Better Batches, Better Privacy: Random Allocation for DP-BandMF
Random-allocation batching for DP-BandMF cuts RMSE up to 1.85 times and speeds privacy accounting on GPUs.
Closed-Form Noise Calibration Against Membership Inference for Random-Allocation DP-SGD
A closed-form noise formula bounds membership-inference accuracy for random-allocation DP-SGD and needs less noise.
Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?
Untied embeddings beat weight tying for decoder-only LLMs under DP-SGD, raising accuracy and cutting memory by over 60%.
Subgroup Membership Inference Audits of Differentially Private Synthetic Text
Audits of 32 differentially private synthetic-text releases show subgroup membership leakage is concentrated in few records and systematically underestimated by average-case attacks.