EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory
EngramEdit updates facts in DeepSeek Engram conditional memory while keeping the Transformer backbone fixed.
EngramEdit is a method for factual updates in conditional-memory LLMs such as DeepSeek Engram, which looks up learned embeddings from input n-grams. It computes target memory representations so the model predicts the revised fact across expressions, then jointly updates shared n-gram embeddings while penalizing changes to frequently reused embeddings. Experiments report near-perfect edit success, use of revised knowledge on unseen expressions and multi-hop reasoning, and nearly three times the strongest baseline's accuracy under chain-of-thought prompting, with unrelated knowledge largely preserved.
- EngramEdit edits shared n-gram embeddings while the Transformer stays fixed.
- Target memories are computed so updated facts hold across multiple expressions.
- Frequent embeddings are penalized more to limit collateral knowledge change.
- Edited facts transfer to unseen phrasing and multi-hop chain-of-thought reasoning.
- Success is near-perfect and about three times the strongest baseline under CoT.
- Organizations
- DeepSeek
- AI models
- DeepSeek Engram
From the original article
1 min read at huggingface.co
Conditional memory architectures such as DeepSeek Engram use input n-grams to look up learned embeddings, expanding the capacity of large language models (LLMs) with limited additional computation.
By Hongru Cai, Ran Wei, Wenjie Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.