ROSETTA: Efficient and Accurate Privacy-Preserving LLM Decoding via Hybrid CKKS/TFHE Evaluation
ROSETTA is a hybrid CKKS/TFHE homomorphic encryption framework for privacy-preserving LLM decoding, achieving up to 4.8x Softmax and 2.1x end-to-end speedups.
The paper proposes ROSETTA, a hybrid CKKS/TFHE fully homomorphic encryption framework for private inference on generative LLMs, targeting the nonlinear operations that dominate autoregressive decoding cost. It introduces an adaptive segmented lookup-table protocol based on TFHE and a scheme-aware operator-selection framework that assigns each nonlinear operator to CKKS or TFHE to minimize latency. Experiments show up to 4.8x Softmax speedup and 1.5-2.1x end-to-end decoding speedup over the state-of-the-art CacheMir framework.
- Hybrid CKKS/TFHE scheme accelerates nonlinear operations in FHE-based LLM decoding
- Adaptive segmented lookup-table protocol enables efficient, accurate nonlinear evaluation
- Scheme-aware operator selection assigns each operator to CKKS or TFHE automatically
- Up to 4.8x Softmax speedup and 1.5-2.1x end-to-end speedup over CacheMir
From the original article
1 min read at arxiv.orgGenerative large language models (LLMs) have achieved state-of-the-art performance on many real-world tasks such as code generation and question answering. These models predominantly rely on an autoregressive decoding strategy that generates output tokens sequentially.
By Jiangrui Yu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.