VoxMem: Benchmarking Multimodal Memory in Large Audio Language Models
VoxMem finds no large audio language model exceeds 40% memory accuracy at 32K across 177 hours of speech.
VoxMem evaluates multimodal memory in large audio language models with 3,196 instances over 34,743 spoken sessions totaling 177 hours. It pairs four acoustic evidence types—speech semantics, speaker identity, paralinguistic cues, and environmental sound—with extraction, multi-session reasoning, temporal tracking, and answer refusal. Budgets range from 8K to 64K tokens. Across 15 LALMs, no model exceeded 40% at 32K, and lexical memory far outperformed speaker, manner, and ambient-sound memory as tasks and histories grew harder.
- 3,196 instances span 34,743 spoken sessions and 177 hours
- Four acoustic evidence types crossed with four memory operations
- Context budgets are stratified from 8K to 64K tokens
- No tested LALM exceeds 40% at 32K context
From the original article
1 min read at huggingface.co
Spoken conversational systems must recover information from prior interactions (i. e. , memory), yet relevant information in speech extends beyond what was said to who said it, how it was spoken, and what was audible, information that exists only in the audio signal and cannot be recovered from a transcript.
By Yang Xiao, Vidhyasaharan Sethu, Eun-Jung Holden · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.