Skip to content
NewA calmer, faster ZeroHour

ScoreCLR estimates contrastive repulsion with denoising score matching

Original title: Contrastive latent representation learning with denoising score matching

arXiv cs.AI / cs.LG / cs.CLby Christopher Marouani ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

A lightweight score network replaces quadratic negative sampling; linear-probe accuracy stays close to SimCLR on CIFAR-10 and ImageNet-100.

The paper introduces ScoreCLR, a contrastive representation method that estimates the repulsive term from the score of the latent representation rather than a large batch of explicit negatives. Denoising score matching trains a lightweight network jointly with the encoder, with noise-level regularization and a Gaussian prior on the latent space. The repulsion cost is linear in batch size instead of quadratic. Under a shared recipe, linear-probe accuracy on CIFAR-10 and ImageNet-100 is close to SimCLR, while projector outputs have higher effective rank and sit closer to the Gaussian prior.

Key facts
  • Repulsion is estimated from the score of the latent representation.
  • Cost scales linearly with batch size rather than quadratically.
  • Compared with SimCLR under a shared training recipe.
  • Evaluated on CIFAR-10 and ImageNet-100 linear probes.
  • Projector outputs show higher effective rank than SimCLR.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Contrastive representation learning methods operate by pulling different views of the same data sample together while pushing them away from views of different samples. A fundamental challenge is the estimation of the repulsive force, which typically requires large batch sizes, can exhibit high variance, and is susceptible to the curse of dimensionality.

arXiv cs.AI / cs.LG / cs.CL

By Christopher Marouani · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.