Skip to content
NewA calmer, faster ZeroHour

Embedding cosine similarity mostly tracks length, not AI homogenization

Original title: Embedding Cosine Measures Text Length Before It Measures Homogenization

arXiv cs.AI / cs.LG / cs.CLby Fengming Liu ()1 min read1 · 1 read
AI summary · x-ai/grok-4.7Summaries by Vani AI

Length controls reproduced most of a reported post-2022 political asymmetry, and corrected measures show the camps stopped converging before ChatGPT.

The paper argues that mean pairwise cosine similarity of mean-pooled encoder embeddings largely measures text length when the encoder was not trained for sentence similarity. On 6.06 million Reddit comments, truncating text and length-matched placebos account for most of an apparent conservative shift toward homogeneity. On 575,000 Stack Exchange answers, a reported post-ChatGPT technical homogenization is fully reproduced by a length placebo. A census of 31 studies locates the risk in anisotropic encoders with unequal lengths or time series.

Key facts
  • Truncating comments from 10 to 150 words raised mean cosine by eight times the reported break.
  • An equal-length random swap reproduced 81 percent of the asymmetry; a fixed-content simulation reproduced 78 percent.
  • The leading principal direction carried 99.9 percent of raw DeBERTa homogeneity.
  • After length correction the asymmetry disappeared under every encoder.
  • Cross-camp replies engaged their targets more closely than same-camp replies.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Mean pairwise cosine similarity between transformer embeddings is a standard measure of whether online discourse is converging under generative AI. When the embeddings are mean-pooled outputs of an encoder never trained for sentence similarity, this measure mostly tracks text length. In 6.

arXiv cs.AI / cs.LG / cs.CL

By Fengming Liu · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.