Coca-Cola: Latent context compression for LLM agents
Original title: Tools Need Not Speak in Text: Latent Context Compression for Agents
Coca-Cola compresses tool observations into latent tokens and embeds them into the input embedding space of LLM agents.
The paper challenges the assumption that LLM agents must consume tool outputs as explicit text tokens. Coca-Cola compresses observations into latent tokens and uses Action-Preserving Training to align next-action distributions. Experiments on web-search agents across four QA benchmarks show 70-89% token reduction and improved performance with two model families. Code is available on GitHub.
- 70.01%-89.24% token consumption reduction
- 24.45%-55.97% end-to-end runtime reduction
- Action-Preserving Training introduced
- Works with four QA benchmarks
- Code at github.com/THU-Agent/Coca-cola
- Organizations
- THU-Agent
From the original article
1 min read at huggingface.co
Large language model (LLM) agents typically append raw tool outputs to their context. Over multi-step interactions, long and noisy observations incur substantial context overhead and dilute the agent's attention to task-relevant information. Existing token pruning and summarization methods remain constrained by discrete text representations, limiting the achievable compression ratios.
By Yuetong Liu, Zhehao Zhou, Yitong Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.