Towards TEE-Certified DP: Verifiable Differentially Private Training on Legacy GPUs
Framework uses CPU TEEs with probabilistic checking to verifiably enforce differential privacy on GPU-offloaded gradient computation at modest overhead.
The paper proposes verifiable differentially private training using CPU-side TEEs combined with untrusted GPUs, targeting legacy hardware that lacks efficient multi-GPU TEE support. Gradient computation is offloaded to GPUs while the CPU TEE verifies correct DP enforcement on gradients through probabilistic checking, avoiding the prohibitive overhead of zero-knowledge proof approaches. The framework detects frequent full deviations from DP with high probability, and evaluated forged-gradient attacks show sparse deviations provide limited utility benefit with no measurable additional membership leakage. Experiments show only modest overhead compared with standard GPU-based DP training.
- Framework offloads gradient computation to untrusted GPUs while a CPU TEE verifies DP enforcement probabilistically.
- Targets legacy GPUs where multi-GPU TEE support is absent or inefficient.
- Detects frequent full DP deviations with high probability at modest overhead.
- Evaluated forged-gradient attacks show sparse deviations give limited utility and no extra membership leakage.
Full article258 words · extracted from arxiv.org · click to collapse
Wide adoption of machine learning has created growing policy and regulatory demand for protecting sensitive training data, with differential privacy (DP) emerging as a key mechanism. Yet a less-studied problem is how to certify the faithful execution of DP during training: an external verifier should be able to check that a released model was trained with proper DP protection, without accessing the private training data. Existing cryptographic approaches, such as zero-knowledge proofs, provide strong guarantees but often incur prohibitive overhead, in some cases by orders of magnitude. Trusted Execution Environments (TEEs) offer a more efficient alternative, but the multi-GPU TEE support needed for training and fine-tuning large language models remains limited to recent platforms and is absent or inefficient on legacy GPUs. To address this, we propose a practical framework for verifiable DP training using CPU-side TEEs together with untrusted GPUs. Our design addresses a fundamental efficiency-security tension: training entirely inside a CPU TEE is too slow, while unrestricted GPU offloading can allow malicious deviations from DP. We therefore offload expensive gradient computation to GPUs, while using the CPU TEE to efficiently verify the correct enforcement of DP on gradients through probabilistic checking. Our framework detects frequent full deviations from DP with high probability; for the utility-oriented forged-gradient attacks evaluated in this work, sparse deviations provide limited utility benefit and show no measurable additional membership leakage. Experiments further show that our approach nearly achieves a ``free lunch'': it incurs only modest overhead compared with standard GPU-based DP training, while effectively constraining malicious deviations from the claimed DP execution.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.20532