The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding
Researchers introduced Brain2Semantics2Text, decoding sentence meaning from non-invasive MEG brain recordings via a semantic bottleneck, improving on prior Brain2Text methods.
The paper proposes Brain2Semantics2Text, a non-invasive speech decoding method that maps sentence-level magnetoencephalography (MEG) responses into a semantic embedding space and inverts those embeddings into natural language. Motivated by evidence that high-level semantic representations are distributed across cortex and evolve on slower timescales, the approach targets meaning rather than phonemes or words, avoiding the need for word-level alignment. The authors report improved sentence-level results compared to prior non-invasive Brain2Text methods despite the low signal-to-noise ratio of neural recordings.
- Decodes text from MEG via an intermediate semantic embedding space
- Avoids word-level alignment by targeting high-level meaning
- Shows improved sentence-level results vs. prior Brain2Text methods
Full article143 words · extracted from huggingface.co · click to collapse
Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.10296