Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

LoopVQ Enables Sub 1-Bit KV Cache Quantization for Looped Transformers

2 sources · 2 articlesFirst reported Updated ()1
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

LoopVQ jointly quantizes channels across loop steps for sub 1-bit KV cache in looped transformers while preserving accuracy and halving decode latency at 32K context.

Looped transformers reuse layers for efficiency but store growing KV cache per step. LoopVQ codes channels jointly across multiple loop steps and fine-tunes codebooks. At 0.5 bits per element it outperforms existing methods on long chain-of-thought and retrieval tasks. Combined with TaSQ it retains BF16 accuracy at 0.25 bits per element. The approach nearly halves decode latency at 32K context and raises peak serving throughput by up to 4.3x on 1.4B and 6.1x on 2.6B Ouro models.

Key facts
  • Joint channel coding for multiple loop steps
  • 0.5 bits per element outperforms existing KV cache methods
  • Retains BF16 accuracy at 0.25 bits with TaSQ
  • Halves decode latency at 32K-token context
  • Up to 4.3x and 6.1x throughput on 1.4B and 2.6B Ouro models
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

What each outlet adds

beyond the shared facts
  • Hugging Face daily papersOutperforms on general long chain-of-thought and retrieval tasks, plus SGLang latency halving
  • arXivPeak throughput gains of 4.3x and 6.1x on 1.4B and 2.6B Ouro models
Products
SGLang
AI models
OuroHuginn

Coverage

oldest first · one row per article · UTC
  1. · 6d ago Hugging Face daily papersFirst report

    LoopVQ sub-1-bit KV cache quantization for looped transformers

    LoopVQ jointly quantizes channels across multiple loop steps in looped transformers to reduce memory while preserving accuracy.

  2. · 5d ago arXiv cs.AI / cs.LG / cs.CL

    LoopVQ enables sub 1-bit KV cache quantization for looped transformers

    LoopVQ quantizes KV cache at 0.5 bits per element for looped transformers, nearly halving decode latency at 32K context.