Linkup Research Releases SPARSEUP: A 149M-Parameter Open-Source Sparse Embedding Model
Linkup Research released SPARSEUP, a 149M-parameter open-source sparse embedding model under Apache 2.0, scoring 56.4 nDCG@10 on BEIR-13.
Linkup Research released SPARSEUP, an open sparse embedding model built on a 149M-parameter ModernBERT backbone with weights available on Hugging Face under Apache 2.0. It was fine-tuned from LightOn's LateOn checkpoint using contrastive learning with hard negatives, adding a log(1+ReLU(x-15)) logit shift, per-token top-12 expansion, and byte-BPE case folding to control sparsity. Linkup reports 56.4 average nDCG@10 on BEIR-13, ahead of opensearch-neural-sparse (54.6) and splade-v3 (51.7), though it trails LightOn's DenseOn (57.9) and LateOn (58.9) under identical backbone and data. With a Seismic inverted index it achieves over 97% recall versus exact search at roughly 380 microseconds per query on MS MARCO.
- 149M-parameter ModernBERT backbone, weights on Hugging Face under Apache 2.0
- 56.4 avg nDCG@10 on BEIR-13, claimed best public sparse encoder under 150M params
- Three fixes: logit shift of 15, per-token top-12 expansion, and case folding
- Trails LightOn's DenseOn by 1.52 and LateOn by 2.5 points with identical data
- Seismic index delivers over 97% recall at ~380 microseconds per query
- AI models
- SPARSEUP
From the original article
3 min read at marktechpost.com
Linkup Research has released SPARSEUP, an open-source sparse embedding model built on a 149M-parameter ModernBERT backbone. It scores 56. 4 nDCG@10 on BEIR-13, which Linkup calls the best result it knows of for a public sparse encoder under 150M parameters.
By Asif Razzaq · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.