Sable proves bit-exact quantized LLM inference on one GPU
First merged summary.
Sable, a single-GPU sumcheck argument, proves bit-exact integer-quantized LLM inference; proofs are public, not zero-knowledge, and a CPU verifier checks them.
Sable is a single-GPU argument system that proves a committed integer-quantized transformer produced a given token sequence bit for bit. It streams layers on one NVIDIA RTX PRO 6000 Blackwell GPU and covers Qwen2.5-32B, Qwen3-32B, and mixture-of-experts Qwen3-30B-A3B. Qwen2.5-32B takes 92.5 seconds for 4,096 tokens, about 6.4 cents per request, and 398 seconds for 16,384 tokens; Qwen3-32B and Qwen3-30B-A3B take 121.8 and 84.7 seconds at 4,096 tokens. Proofs are not zero-knowledge, target open-weight models, cost 41 to 50 times serving time, and a CPU verifier checks them in 19 to 56 seconds. The authors report soundness error 2^-103 in the random-oracle model and say the code is public.
- Qwen2.5-32B takes 92.5 seconds for 4,096 tokens and 398 seconds for 16,384.
- Qwen3-32B and Qwen3-30B-A3B take 121.8 and 84.7 seconds at 4,096 tokens.
- Proofs cost 41 to 50 times serving time and are 40 to 64 times faster than zkLLM per parameter and token.
- Greedy tokens match BF16 at 86.3% of WikiText-2 positions.
- Authors report soundness error 2^-103 in the random-oracle model; code is public.
- A CPU verifier checks proofs in 19 to 56 seconds.
What each outlet adds
beyond the shared facts- arXiv cs.CRSable single-GPU proof times, cost, verifier range, and claimed soundness.
Coverage
oldest first · one row per article · UTC- · 1d ago
arXiv cs.CRFirst report
Second-order screen predicts quantization mismatch on five language models
On unseen Mistral-7B-v0.3, all six predictions fell in band and vLLM throughput was predicted within 0.4%.
- · 15h ago
Sable proves bit-exact Qwen inference on one GPU
On one RTX PRO 6000, Sable proves Qwen2.5-32B in 92.5 seconds for 4,096 tokens, about 6.4 cents per request.