Latent Compaction with Test-Time Training
Latent Compaction keeps 96.7% RULER accuracy at 32x KV-cache compression via test-time training.
Latent Compaction learns a compact document-specific key-value cache via test-time training while model weights stay frozen. Cache entries are initialized from reconstruction importance, teacher attention, and key-space diversity, then distilled from the full-cache model using only generic prompts. On RULER it reaches 96.7% accuracy at 32x compression, compared with 40.0% for KVZip+, with further gains on LongBench and Phonebook Lookup. The method transfers across scales and families, including retrieval gains on Qwen3 and MiniCPM5.
- Latent Compaction optimizes a document-specific KV cache with frozen model weights.
- Training uses generic document prompts, not downstream questions or answers.
- On RULER it reaches 96.7% accuracy at 32x compression versus 40.0% for KVZip+.
- Gains transfer to Qwen3 and MiniCPM5; code is released.
- Organizations
- THUNLP
From the original article
1 min read at huggingface.co
Long contexts enlarge key--value (KV) caches, increasing memory use and attention costs during generation. Compressing these caches must preserve information for unseen questions. We introduce **Latent Compaction**, which learns a compact document-specific cache through test-time training.
By Yingfa Chen, Xingyu Shen, Chaojun Xiao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.