SKIM stores dialogue turns inside frozen language-model parameters
Original title: Self-Keyed In-Parameter Memorization for External Memory-Free Dialogue Recall in Frozen Language Models
The model writes self-generated questions as retrieval keys and later answers user questions from parameters alone, with no external store.
Self-keyed in-parameter memorization (SKIM) is a closed-form method for recalling dialogue when a frozen language model has no external memory and no advance knowledge of the question. For each turn, the model generates its own questions and writes them into its parameters as keys that later retrieve that turn. Many varied questions per turn are meant to match differently phrased user questions. On five language models, the authors say SKIM recalls most stored turns and answers from parameters without external storage.
Unconfirmed: The paper does not name the five models or quantify recall beyond "most" stored turns.
- The setting assumes no external memory and no prior knowledge of the question.
- Multiple generated questions per turn are intended to handle rephrasing.
- The paper does not name the five evaluated language models.
From the original article
1 min read at arxiv.orgA large language model maintains conversational content in an external storage that is outside the model and grows as the dialogue continues. This work assumes an environment where a language model has no external memory and no prior knowledge of the future question or its answer. The motivation is to research whether a dialogue can be stored in the model parameters and later answer a question about it.
By XinYu Piao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.