MemoryAthena: Adaptive Routing over Latent and Generated Memories
MemoryAthena routes generated memory as a bounded correction to direct Engram retrieval.
MemoryAthena tests whether useful memory can be generated as well as retrieved, using direct Engram retrieval, generation from retrieved Engram cues, and generation from causal backbone states. With the backbone, memory, generators, and readers frozen, a lightweight causal routing head learns when generated memory should modify the direct-retrieval residual through bounded interpolation. Question-answering performance rises from 37.65 to 39.28 on a five-task average, while a six-task general-NLP average rises from 76.73 to 79.13. The complete memory-side system contains about 201 million parameters, excluding the frozen backbone.
- Pathways cover direct retrieval and two generation modes
- A frozen-backbone routing head uses counterfactual token advantages
- Five-task QA average rises from 37.65 to 39.28
- Memory-side system contains about 201 million parameters
From the original article
1 min read at huggingface.co
Learned-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be modified independently. We study whether useful memory can also be generated rather than only retrieved.
By Mingyuan Li, Guangsheng Yu, Juyuan Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.