SpliTEE: Improving LLM Inference on Trusted Hardware with Differentially Private GPU Outsourcing
SpliTEE splits LLM inference between Intel TDX trusted execution and untrusted GPUs, using differential privacy instead of encryption to protect intermediate representations.
SpliTEE extends split inference to LLMs, running inference partly inside an Intel TDX TEE while masking intermediate inputs sent to untrusted GPUs with differential privacy rather than encryption. The authors show a prompt-reconstruction attack recovers nearly 80% of prompts from unmasked intermediate representations, motivating the masking. A global sensitivity analysis bounds the required DP noise scale, avoiding quantization and keeping models in floating point. The implementation is nearly twice as fast as full CPU-based TDX inference and 5-15 seconds faster than encryption-based Slalom with higher accuracy, evaluated on Llama-3.2-3B and Qwen3-4B.
- Prompt-reconstruction attack recovers ~80% of prompts from unmasked intermediate representations
- Differential privacy replaces encryption, avoiding quantization and keeping floating-point inference
- Nearly 2x faster than CPU-only TDX; 5-15s faster than Slalom with higher accuracy
- Evaluated with Intel TDX on Llama-3.2-3B and Qwen3-4B
From the original article
1 min read at arxiv.orgUser prompts provided to large language models (LLMs) may contain sensitive or private information that can be misused by remotely deployed models, such as through inadvertent memorization during retraining.
By Shashie Dilhara Batan Arachchige · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.