DP-ES: Differentially Private Evolution Strategies for Prompt Optimization
DP-ES privately optimizes full prompts and beats DP-OPT by 38.6 points on GSM8K under a tight privacy budget.
The paper proposes DP-ES, which evolves a population of complete prompts using LLM mutations that never see private data and spends privacy only on sampled-Gaussian evaluation. Under a conservative epsilon at most 1.0 and delta of 10^-5, it scores 88.1% on GSM8K versus 49.5% for DP-OPT, with about nine times lower variance, plus 99.7% on MedQA, 73.5% on BANKING77, and 86.8% on Alpaca. It is 2.5 times faster and uses 3.3 times fewer private-data call groups. End-to-end tests on genuinely sensitive deployment data remain future work.
- Greedy token-level DP-OPT is unstable under tight privacy budgets.
- Privacy is spent only on Gaussian evaluation; mutations never touch private data.
- GSM8K rises from 49.5% to 88.1% at epsilon at most 1.0.
- Also reports MedQA 99.7%, BANKING77 73.5%, and Alpaca 86.8%.
From the original article
1 min read at arxiv.orgToken-level differentially private (DP) prompt optimization methods such as DP-OPT can become unstable under tight privacy budgets: on GSM8K, DP-OPT obtains $49. 5\pm28. 5\%$ across 30 runs, and a logged search trajectory reveals prompt-template drift and noise-sensitive irreversible choices.
By Ziniu Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.