MemT-1.0 releases Nano and Flash memory-native models
Original title: MemT-1.0: Towards Memory-Native Evolution
At 128K-token history, Nano is about 1.86 times faster and uses 66 times less persistent storage than a full-context KV cache.
MemT-1.0 is a memory foundation-model architecture that treats memory as part of model computation, built on Metis with Gated DeltaNet-2 and Mixture-of-Storage. Training uses continued pre-training and post-training on 34B tokens. The released family, MemT-1.0-Nano and MemT-1.0-Flash, beats parametric memory baselines on average, stays competitive with retrieval-augmented methods, and largely keeps general capability. Internalizing retrieved procedural experience also raises success on agentic tasks.
Unconfirmed: Performance claims come from the authors' own evaluation and are not independently confirmed here.
- Gated DeltaNet-2 updates memory at token level within each turn.
- Mixture-of-Storage is intended to scale native memory capacity.
- Authors report average gains over parametric memory baselines.
- Results are competitive with retrieval-augmented methods while keeping general skills.
From the original article
1 min read at arxiv.orgMemory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1. 0, a new architecture of memory foundation models that internalize memory as part of model computation.
By Zhiyu Li · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.