Seal, Then Sample: Sampled Layerwise Proofs for Verifiable LLM Inference from GPT-2 to 70B
Researchers propose Sampled Layerwise Proofs to audit outsourced LLM inference through Llama-2-70B.
The paper presents Sampled Layerwise Proofs, a protocol that commits every chunk of an LLM inference trace before any challenge and then proves a verifier-selected subset plus the chunks that bind the prompt and answer. On TinyLlama-1.1B, proving 7 of 47 chunks took 22.0% of the time and 6.8% of the proof size of proving all 47. A Llama-2-70B run on a 2 TB CPU host sealed 163 chunks, proved five, and produced a 4.34 MiB proof in 1,259 seconds, verified in 46.3 seconds without the weights. A fixed-point canonical model reached 84.8-84.9% argmax agreement with the floating-point reference over 334,705 WikiText-2 positions. Guarantees cover only the proven chunks.
- SLP commits every inference chunk before the verifier samples what to prove
- Proving 7 of 47 TinyLlama chunks took 22% of full-proof time
- Twelve packed requests were proved 6.5 times faster than separate proofs
- A Llama-2-70B run sealed 163 chunks and proved five on one CPU host
- Fixed-point argmax agreement with floating point was about 84.8%
Full article305 words · extracted from arxiv.org · click to collapse
Verifying outsourced language-model inference requires a precisely identified computation and an audit whose cost a service can afford. We present Sampled Layerwise Proofs (SLP), a protocol and prototype that commits the boundary activations of every chunk of an inference trace, absorbs all commitments before any challenge is drawn, and then proves a verifier-selected subset of chunks together with the chunks that bind the prompt and the answer. Audit coverage becomes a runtime parameter over one set of commitments: on a TinyLlama-1.1B trace, proving seven of 47 chunks takes 22.0% of the time and 6.8% of the proof size of proving all 47. Because proof cost is dominated by weights rather than tokens, SLP packs concurrent requests into one trace under a block-diagonal causal mask and binds the prompt and answer of each request to its slot. Twelve packed requests are proved in 181.9 s, 6.5 times less than twelve separate proofs at the measured single-proof cost, and a simulated service proves twelve requests at 30.6 s per request with 0.6 s of verification each, rejecting a tampered answer. Disk-backed integer weights and streamed polynomial commitments let a single Llama-2-70B run complete on a 2 TB CPU host: 163 chunks sealed, five proved, a 4.34 MiB proof in 1,259 s, verified in 46.3 s without the weights. The proven object is a fixed-point canonical model; we trace a severe fidelity loss to the residual-stream bit width, repair it with an LLM-aware observer, and measure 84.8-84.9% argmax agreement with the floating-point reference over 334,705 WikiText-2 test positions. The limits are stated as precisely: guarantees cover proven chunks only, a fixed invalid chunk in the 70B setting is covered with probability 3/161, a manifest-only Fiat-Shamir schedule can be ground at 12.5 ms per attempt and needs an externally ordered challenge, and all measurements use a test reference string.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.27367