Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

Sable proves bit-exact LLM inference on one GPU

2 sources · 4 articlesFirst reported Updated ()
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

Sable proves a committed integer-quantized open-weight transformer produced a token sequence bit for bit, in tens of seconds on one GPU.

Researchers present Sable, a single-GPU argument system that a committed integer-quantized transformer produced a given token sequence bit for bit. It uses a fixed integer function with no floating-point step and a sumcheck constraint system over the 31-bit KoalaBear prime, streaming layers on one NVIDIA RTX PRO 6000 Blackwell GPU. It covers open-weight Qwen2.5-32B, Qwen3-32B and Qwen3-30B-A3B at 41 to 50 times serving time. Proofs are not zero-knowledge. A CPU verifier checks them in 19 to 56 seconds, and the authors report soundness error of 2^-103.

Key facts
  • Qwen2.5-32B: 92.5 seconds for 4,096 tokens and 398 seconds for 16,384.
  • Qwen3-32B and Qwen3-30B-A3B take 121.8 and 84.7 seconds at 4,096 tokens.
  • A 4,096-token proof is priced at 6.4 cents at public GPU rates.
  • CPU verification takes 19 to 56 seconds; soundness error is 2^-103.
  • Greedy tokens match BF16 at 86.3% of WikiText-2 positions.
  • Proofs are 40 to 64 times faster per parameter and token than zkLLM; code is public.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

What each outlet adds

beyond the shared facts
  • Hugging Face daily papersKoalaBear field, no floating-point step, and public-rate proof pricing.
  • arXiv cs.CR40–64× faster than zkLLM per parameter and token; code is public.
Vendors
NVIDIA
Products
SableRTX PRO 6000 BlackwellvLLM
AI models
Qwen2.5-32BQwen3-32BQwen3-30B-A3BLlama-3-8BQwen3-8BQwen3-4BOLMo-2-7BMistral-7B-v0.3

Coverage

oldest first · one row per article · UTC
  1. · 5d ago Hugging Face daily papersFirst report

    MACaQ quantizes weights and activations to MXFP4 layer by layer

    On Qwen3-32B in vLLM it adds 2.3% overhead, and a 122B model was quantized in 64 minutes using 32.5GB GPU RAM.

  2. · 2d ago arXiv cs.CR

    Second-order screen predicts quantization mismatch on five language models

    On unseen Mistral-7B-v0.3, all six predictions fell in band and vLLM throughput was predicted within 0.4%.

  3. · 2d ago Hugging Face daily papers

    Sable proves bit-exact quantized LLM inference on one GPU

    Qwen2.5-32B takes 92.5 seconds for 4,096 tokens; proofs are not zero-knowledge.

  4. · 1d ago arXiv cs.CR

    Sable proves bit-exact Qwen inference on one GPU

    On one RTX PRO 6000, Sable proves Qwen2.5-32B in 92.5 seconds for 4,096 tokens, about 6.4 cents per request.