Saidzoda fine-tune lifts Belebele recall for four Central Asian languages
Original title: Saidzoda-CA-EmbeddingGemma-300M: One Small Embedder for Tajik, Kazakh, Uzbek and Kyrgyz
Belebele recall@1 rises 14.4 to 26.0 points, to 75.2%–77.7%, while English and Russian recall fall.
Released EmbeddingGemma-300M ranks the correct Belebele passage first for only 51.7% to 60.8% of questions in Tajik, Kazakh, Uzbek and Kyrgyz. Authors fine-tuned it for one more epoch on 279,904 pairs, including synthetic questions, a Tajik refresher, and English and Russian replay. The combined model stays within 2.8 recall@1 points of same-setting single-language models in three languages, but trails larger released XLM-RoBERTa embedders by 9.4 to 10.2 points.
Unconfirmed: The paper does not state whether the fine-tuned weights are publicly released.
- Tajik law recall@10 stays 92.2 against a 92.4 starting point.
- Law recall@1 falls 4.4 points from the starting checkpoint.
- Russian and English recall@1 fall by 3.0 and 4.2 points.
- Scores for released comparison models match published MTEB results within 2.4 points.
From the original article
1 min read at arxiv.orgThe small open embedder EmbeddingGemma-300M, used as released, finds the right passage first for only 51. 7% to 60. 8% of Belebele questions in Tajik, Kazakh, Uzbek and Kyrgyz.
By Tohir Saidzoda · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.