CA-DPSGD uses curvature-aware clipping to reduce DPSGD bias
Original title: Curvature-Aware Clipping: Mitigating Bias in DPSGD via Approximate Second-Order Information
The method pairs a rectified curvature estimator with anisotropic noise and reports better utility than prior clipping methods.
The paper proposes Curvature-Aware Differentially Private Stochastic Gradient Descent, or CA-DPSGD, to reduce geometric misalignment from gradient clipping in differentially private training. It builds a clipping space from approximate second-order information using a rectified curvature-matrix estimator and adds anisotropic noise to keep privacy guarantees. The authors say experiments outperform existing clipping methods. No deployment incident or affected product is described.
- CA-DPSGD targets clipping bias rather than only DP noise.
- Privacy is maintained with an anisotropic noise mechanism.
- The paper reports gains over prior clipping methods but gives no deployment details.
From the original article
1 min read at arxiv.orgDifferentially Private Stochastic Gradient Descent (DPSGD) has become the de facto standard for training deep neural networks with rigorous privacy guarantees. However, the utility of DPSGD is constrained by gradient clipping bias and DP noise. While existing work has tried to address these issues, the geometric misalignment caused by clipping bias remains a critical challenge.
By Xincheng Xu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.