Maverick: Private and Verifiable LLM Inference Made Practical via Matrix-Vector Multiplication Delegation
Maverick protocol delivers private and verifiable LLM inference via matrix-vector multiplication delegation, achieving up to 45x throughput gains over local inference on Qwen3-4B.
Maverick introduces an information-theoretically sound protocol for delegating matrix-vector multiplication with transparent preprocessing, efficient batch verification, and virtually no server overhead, combined with LPN-based pseudorandom masking for input privacy. It addresses privacy and correctness concerns when users delegate open-weight LLM inference to third-party providers. An end-to-end prototype evaluated on Qwen3-4B achieved throughput gains over local inference of up to 45x with precomputed privacy masks and 44x for verification-only workloads, with a CPU server using up to 128 threads.
- First information-theoretically sound verification protocol for matrix-vector multiplication delegation
- LPN-based pseudorandom masking provides client input privacy
- Up to 45x throughput gain over local inference on Qwen3-4B
- Targets privacy and correctness when outsourcing open-weight LLM inference
From the original article
1 min read at arxiv.orgOpen-source large language models (LLMs) are increasingly competitive with closed-source models while offering transparency and the ability to run inference without exposing user inputs to a service provider. However, running large-scale models locally requires substantial computational resources. In practice, users may still resort to a third-party provider, giving rise to privacy and correctness concerns.
By Ben Merbaum · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.