BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
BeaconKV introduces training-free KV cache compression using beacon queries, cutting long-reasoning inference memory up to 5.8x while preserving accuracy.
The paper shows recency-based KV cache compression assumptions fail in long-horizon reasoning because Thought Revisiting Tokens (TRT) re-attend to distant context such as early task-solving plans. TRT queries cluster into a small number of similarity groups, which BeaconKV exploits by maintaining compact beacon query representatives to anticipate revisited KV pairs without storing full query history. The training-free method achieves up to 5.8x memory reduction and over 4.3x throughput improvement across four open-source large reasoning models while nearly preserving full cache accuracy.
33