Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability
A controlled study finds agent memory portability varies sharply: fixed-schema knowledge graphs survive model swaps while compressed notes degrade.
The study compares preserving an agent's history as raw long context, RAG chunks, compressed natural-language notes, or fixed-schema knowledge graphs across model upgrades, using 48 synthetic histories and two open-weight sub-10B-parameter models. Fixed-schema KG accuracy changed by only +0.0004 ± 0.0020 after a writer swap, while compressed NOTES shifted asymmetrically by +9.91 or -13.28 percentage points depending on migration direction. Mixed 50/50 embedding migrations captured only 4.96 of an 11.90-point RAG re-embedding gain; 80% of the NOTES deficit came from information lost at construction, and 81% of the RAG deficit from retrieval failures. Store-only repair of NOTES failed to reach 90% recovery in all 48 cases, while retaining raw histories enabled recovery in 34 of 48 for one direction.
- Fixed-schema knowledge graphs transfer across model swaps with near-zero accuracy drift.
- Compressed NOTES show high model coupling: +9.91 or -13.28 points depending on direction.
- Partial 50/50 embedding migration captures only 4.96 of an 11.90-point RAG gain.
- Authors recommend migration testing, embedding space isolation, and retaining raw source histories.
From the original article
1 min read at arxiv.orgModel upgrades are routine; memory migrations are not. An agent can keep the same memory store and still forget: a new model may interpret old notes differently, mixed embedding versions may break retrieval, and repair may fail without the original evidence.
By Ankit Goyal · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.