Seal, Then Sample: Sampled Layerwise Proofs for Verifiable LLM Inference from GPT-2 to 70B
Researchers propose Sampled Layerwise Proofs to audit outsourced LLM inference through Llama-2-70B.
The paper presents Sampled Layerwise Proofs, a protocol that commits every chunk of an LLM inference trace before any challenge and then proves a verifier-selected subset plus the chunks that bind the prompt and answer. On TinyLlama-1.1B, proving 7 of 47 chunks took 22.0% of the time and 6.8% of the proof size of proving all 47. A Llama-2-70B run on a 2 TB CPU host sealed 163 chunks, proved five, and produced a 4.34 MiB proof in 1,259 seconds, verified in 46.3 seconds without the weights. A fixed-point canonical model reached 84.8-84.9% argmax agreement with the floating-point reference over 334,705 WikiText-2 positions. Guarantees cover only the proven chunks.
- SLP commits every inference chunk before the verifier samples what to prove
- Proving 7 of 47 TinyLlama chunks took 22% of full-proof time
- Twelve packed requests were proved 6.5 times faster than separate proofs
- A Llama-2-70B run sealed 163 chunks and proved five on one CPU host
- Fixed-point argmax agreement with floating point was about 84.8%
- AI models
- GPT-2TinyLlama-1.1BLlama-2-70B
From the original article
1 min read at arxiv.orgVerifying outsourced language-model inference requires a precisely identified computation and an audit whose cost a service can afford.
By Youki Lim · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.