Building Multilingual Bridges: Data Mixing as the Pillar of Generalization for In-Language Reasoning
Tiny Aya L2-Thinker, a 3.35B model, achieves over 93 percent in-language reasoning across 60 languages via optimized multilingual data mixing; weights released.
The paper studies L2 reasoning, the ability to reason consistently in the language of the user's prompt, approached through SFT data composition and scheduling. Tiny Aya L2-Thinker (3.35B) achieves an in-language reasoning rate above 93 percent across 60 languages on six benchmarks spanning math, commonsense, instruction following, open-ended generation, and cultural reasoning. Findings show generalization to held-out languages comes from broader language coverage, multilingual non-reasoning data, and a strong English reasoning backbone, suggesting reasoning is language-agnostic and transferable without per-language supervision. Model weights and multilingual reasoning data are publicly released.
- Tiny Aya L2-Thinker (3.35B) exceeds 93% in-language reasoning across 60 languages
- Data mixing, coverage, and English reasoning backbone drive held-out generalization
- Reasoning shown transferable without per-language reasoning supervision
- Weights and multilingual reasoning data released publicly
From the original article
1 min read at arxiv.orgReasoning language models have made substantial advances on a variety of complex tasks, yet their capabilities remain overwhelmingly English-centric: models primarily reason in English regardless of the language they are prompted in. This is inaccessible for non-English-speaking users, risks losing the intent of the original question, and forgoes knowledge more readily expressed in the target language.
By Mehrnaz Mofakhami · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.