Homomorphic Advantage Operator: Stabilizing Reinforcement Learning Under Fully Homomorphic Encryption Constraints
A new operator stabilizes deep reinforcement learning when training runs under fully homomorphic encryption constraints.
The paper introduces the Homomorphic Advantage Operator, a stabilization method for deep reinforcement learning when non-linearities must be replaced by polynomials under fully homomorphic encryption. HAO applies a zero-mean centering projection to temporal-difference targets, removing the uniform baseline that causes Bellman drift without extra multiplicative depth or ciphertext bootstrapping. On a tabular MDP, real CKKS-encrypted CartPole, and a 20-node logistics benchmark, HAO agents had 0% approximation-boundary breaches, versus 83.8% of episodes for the unstabilized baseline, and improved tabular optimal-policy accuracy by 18.0 percentage points.
- Polynomial approximations for FHE-based reinforcement learning diverge via recursive Bellman drift.
- HAO centers temporal-difference targets without adding non-linear multiplicative depth.
- Encrypted CartPole and a 20-node routing benchmark had zero boundary breaches.
- HAO raised tabular optimal-policy accuracy by 18.0 percentage points.
From the original article
1 min read at arxiv.orgPrivacy-preserving machine learning presents significant deployment challenges on the cloud for intelligent systems with confidential data. Fully Homomorphic Encryption (FHE) offers a compelling solution for secure computation, preserving data confidentiality of cloud computations.
By Abid Mohamed Nadhir · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.