Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning
Researchers train Tiny Aya L2-Thinker, a 3.35B model achieving over 93% in-language reasoning across 60 languages via multilingual data mixing.
The paper addresses L2 reasoning, where models reason consistently in the language of the user's prompt rather than defaulting to English. Through data-centric SFT optimization, the 3.35B Tiny Aya L2-Thinker reaches above 93% L2 reasoning rate across 60 languages on 6 benchmarks covering math, commonsense, instruction following, open-ended generation, and cultural reasoning. The authors find that generalization to held-out languages relies on broad language coverage, multilingual non-reasoning data, and a strong English reasoning backbone, without needing reasoning supervision in every target language. Model weights and multilingual reasoning data are released.
- 3.35B Tiny Aya L2-Thinker exceeds 93% in-language reasoning across 60 languages
- Performance spans 6 benchmarks including math and cultural reasoning
- Generalization to held-out languages achieved without per-language reasoning supervision
- Released reasoning data includes multilingual non-reasoning datasets
- Findings indicate reasoning is a transferable, language-agnostic behavior
- Products
- Tiny Aya L2-Thinker
- Organizations
- Aya
- AI models
- Tiny Aya L2-Thinker
From the original article
1 min read at huggingface.co
Reasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: models primarily reason in English regardless of the language they are prompted in. This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language.
By Mehrnaz Mofakhami, Ananya Sahu, Alejandro R. Salamanca · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.