Sable proves bit-exact Qwen inference on one GPU
Original title: Sable: A Small-Field Sumcheck Argument for Bit-Exact Inference of Large Language Models
On one RTX PRO 6000, Sable proves Qwen2.5-32B in 92.5 seconds for 4,096 tokens, about 6.4 cents per request.
Sable is a single-GPU argument system that proves a committed integer-quantized transformer produced a given token sequence bit for bit. It streams layers on one NVIDIA RTX PRO 6000 Blackwell GPU and covers Qwen2.5-32B, Qwen3-32B, and mixture-of-experts Qwen3-30B-A3B. Proofs are not zero-knowledge and target open-weight models; a CPU verifier checks them in 19 to 56 seconds.
- Qwen2.5-32B takes 92.5 seconds for 4,096 tokens and 398 seconds for 16,384.
- Qwen3-32B and Qwen3-30B-A3B take 121.8 and 84.7 seconds at 4,096 tokens.
- Proofs cost 41 to 50 times serving time and are 40 to 64 times faster per parameter and token than zkLLM.
- Greedy tokens match BF16 at 86.3% of WikiText-2 positions.
- Soundness error is 2^-103 in the random-oracle model; code is public.
- Products
- Sable
- AI models
- Qwen2.5-32BQwen3-32BQwen3-30B-A3B
From the original article
1 min read at arxiv.orgWe present Sable, an argument system and single-GPU prover for the statement that a committed, integer-quantized transformer produced a given token sequence from a given prompt, bit for bit.
By Tomer Solberg · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.