Conditional Program stores LLM n-gram lookup tables at two bits
Original title: N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
For DeepSeek, the method cuts table storage from 203GB to 51GB versus keeping full embeddings.
Conditional Program compresses conditional n-gram memory so only requested rows are reconstructed, using a short code, a local scale, and a shared reconstruction program. A Gaussian trellis instantiation with two-bit codes lowered output KL by 41–63% relative to 2-bit round-to-nearest. Across three trained n-gram memory families, the shared trellis beat fitted codebooks and low-rank residual variants at matched storage. For DeepSeek, table storage fell from 203GB to 51GB.
- Two-bit codes cut output KL 41–63% versus 2-bit round-to-nearest.
- Analysis covered three trained n-gram memory families.
- Shared trellis beat fitted codebooks and low-rank residual variants tested.
- DeepSeek table storage dropped from 203GB to 51GB.
- Rows stay independently decodable via short codes and local scales.
- Organizations
- DeepSeek
From the original article
1 min read at huggingface.co
Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table.
By Donghoon Han, Minseop Jung, Dong Woo Kwak · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.