Perseus enables fast homomorphic inference for LLMs
Original title: Perseus: A Full-Stack System for Homomorphic Model Inference
Perseus is 3.96x faster than other FHE implementations for Llama-3-8B
Perseus is an open Python library for full-stack homomorphic encrypted model inference. It includes optimizations at backend, data, and model levels achieving state-of-the-art latency within 3% accuracy of plaintext models. It is the first public speedrun on encrypted GPT-2.
- 3.96x faster at single-token decoding for Llama-3-8B
- Within 3% of plaintext accuracy
- Novel optimizations at each stack level
- Supports encrypted nonlinearities and end-to-end execution
From the original article
1 min read at arxiv.orgLarge Language Models (LLMs) have made it exceptionally convenient for users to send their private data to model providers, which process them in plaintext. Fully Homomorphic Encryption (FHE) promises to secure remote inference by computing directly on encrypted data, yet it has remained prohibitively slow.
By Alessandro Zirilli · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.