Memory Tax argues many agent memory systems lose to naive RAG
Original title: Memory Tax: Dissecting the Net Value of Agent Memory
The authors split memory's net value into gains over raw-history retrieval and losses they call memory tax.
The Memory Tax paper evaluates dedicated agent memory against naive retrieval from raw history across multiple benchmarks and model backbones. The authors report that many memory systems underperform naive RAG, which they treat as negative net value. They separate gains, called memory benefit, from losses that remove or obscure evidence, called memory tax, and say they will open-source code and results.
Unconfirmed: Which systems and benchmarks underperformed is not named in the abstract.
- RAG over raw history is the reference for measuring net value.
- Memory can reorganize facts or distill preferences but also hide evidence.
- The project page is https://hexuan21.github.io/MemoryTax/.
- Authors say code and results will be open-sourced.
From the original article
1 min read at huggingface.co
Memory is increasingly used in long-horizon agents, yet the net value of building memory over direct retrieval from raw history remains largely unclear. Through a large-scale evaluation across multiple benchmarks and model backbones, we find many dedicated memory systems underperform naive RAG in many cases, indicating a negative net value.
By Xuan He, Sining Zhoubian, Ruozhen Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.