Perplexity releases pplx-embed-v2-late 0.6B and 9B models
Original title: pplx-embed-v2-late Technical Report
Both models, distilled from an 18B teacher, share an embedding space so the 9B can index and the 0.6B can query.
Perplexity AI published a technical report on pplx-embed-v2-late, a pair of 0.6B and 9B multilingual and multimodal late-interaction embedding models. The report says both reach state-of-the-art results for their size on text, image, and visual document retrieval, and that the 0.6B model is competitive with much larger models on visual documents. They were trained by representation distillation from an 18B teacher and are available on Hugging Face.
- The 0.6B and 9B models share an embedding space distilled from an 18B teacher.
- Documents can be indexed with the 9B model and queried with the 0.6B model.
- The 0.6B model is competitive on visual document retrieval with models several times larger.
- Weights are in the perplexity-ai Hugging Face collection pplx-embed-v2.
- Vendors
- Perplexity AI
From the original article
1 min read at arxiv.orgIn this report, we introduce pplx-embed-v2-late, a pair of 0. 6B and 9B multilingual and multimodal late-interaction models. Both models achieve state-of-the-art results within their respective size classes across a range of text, image, and visual document retrieval benchmarks.
By Markus Krimmel · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.