Embedding Models Measure in Peculiar Ways
Researchers find embedding models only weakly represent physical measurements like mass, distance, time, and volume, dominated by superficial string similarity.
The study tests whether embedding spaces capture objective notions of semantic equivalence and distance defined by physical measurements of mass, distance, time, and volume. Physical measurement is found to be only weakly modeled in embedding space, with representations strongly influenced by superficial string similarity, and recalibrating similarity metrics does not substantially improve alignment.
- Embedding distance poorly tracks objective physical measurement equivalence
- Superficial string similarity strongly shapes how measurements are represented
- Recalibration of similarity metrics yields little improvement
From the original article
1 min read at arxiv.orgEmbedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit a unique, objective notion of semantic equivalence and distance. We find that physical measurement is only weakly modeled in the embedding space, and that instead quite peculiar measurement patterns can be observed.
By Juri Opitz · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.