Stanford open-sources Memorilla, a latent memory module for frozen LLMs
Original title: Stanford Research Open-Sources Memorilla, a Latent Memory Module for Frozen LLMs
A 194M-parameter module gives a frozen Qwen3-8B 16 tokens and beat full context on FactKG with 108x fewer document tokens.
Stanford researchers in Jure Leskovec's SNAP group, with co-authors from Toronto, Penn and MIT, released Memorilla under Apache 2.0. A frozen Qwen3-Embedding-4B encodes documents once; 16 learned slots attend to the question and store, then are injected into a frozen Qwen3-8B. The team reports a 49.68 average across eight benchmarks, the top score on five, with code, recipes and checkpoints public. Scores are self-reported.
Unconfirmed: Benchmark scores are self-reported by the research team.
- Only the memory module trains: 194,123,776 parameters for the 8B decoder.
- FactKG accuracy was 84.33% versus 80.43% with every document in context.
- PersonaMem-v2 was 44.60% versus 27.92% for full context; PubMedQA was 59.00% versus 69.00% for LLMLingua-2.
- Training took about 19 hours on 4 H100 GPUs; the quickstart needs one 80 GB GPU.
- On one H100, Memorilla answered in 0.74 seconds at 512 documents versus 19.9 seconds for full context.
- Organizations
- Stanford UniversitySNAP
From the original article
8 min read at marktechpost.comStanford researchers released Memorilla, an open-source memory system that compresses a document collection into 16 memory tokens for a frozen Qwen3-8B. It reaches 84. 33% on FactKG using 108x fewer document tokens than full context.
By Asif Razzaq · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.