HiVe-Mem cuts multimodal agent memory tokens by about 90%
Original title: HiVe-Mem: Interleaving Horizontal Memory Expansion and Vertical Evidence Composition for Multimodal Agents
A graph of textual and visual anchors plus a trainable evidence selector improved three memory benchmarks across base models.
HiVe-Mem is a multimodal memory framework for agents with long interaction histories. Horizontal expansion stores episodes in a relational graph of textual and visual anchors and grows retrieved seeds with related neighbors. Vertical composition keeps summaries, captions, visual views, raw images, and interaction history, with a lightweight selector choosing an evidence subset per memory. Across three multimodal memory benchmarks it improved task performance while cutting token use by about 90% versus the most efficient competing multimodal baselines.
- Memories keep summaries, captions, visual views, raw images, and history.
- A trainable selector picks an evidence subset independently for each memory.
- Token use fell about 90% versus the most efficient competing multimodal baselines.
From the original article
1 min read at arxiv.orgAs multimodal agents accumulate increasingly long interaction histories, effectively retrieving and utilizing past information becomes critical for long-term interaction. Existing memory systems typically rely on semantic retrieval over compressed memories and expose predefined representations, which may miss relevant memories and overlook heterogeneous evidence needs across queries.
By Haozhen Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.