All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts
ScriptMoE recognizes scene text across 229 languages, beating per-script models and a vision-language baseline.
Researchers introduce ScriptMoE, an all-in-one multilingual scene-text recognizer that shares one visual encoder and uses a sparse, script-aware mixture-of-experts decoder. Training uses TextMuSS-10M, a synthetic dataset spanning 10 scripts and 229 languages. On TextMuSS-Bench (10,899 images) it scores 82.06%, 1.31 points above the strongest scene-text baseline. Replacing the recognizer in PP-OCRv5 lifts CC-OCR end-to-end F1 from 65.71% to 80.89%, slightly above the best vision-language model at far fewer parameters.
- TextMuSS-10M covers 10 scripts and 229 languages with synthetic scene text.
- ScriptMoE routes each image to the top-2 script-aligned experts plus a shared expert.
- Accuracy reaches 82.06% on TextMuSS-Bench, 1.31 points above the best STR baseline.
- Swapping ScriptMoE into PP-OCRv5 raises CC-OCR F1 from 65.71% to 80.89%, slightly above the best VLM.
- Products
- TextMuSS-10MTextMuSS-Bench
From the original article
1 min read at huggingface.co
Multilingual scene text recognition (STR) remains challenging due to the scarcity of training data for most languages and the difficulty of serving diverse scripts within a single model. Existing solutions either deploy one recognizer per language, inflating cost and introducing error accumulation, or rely on massive vision-language models (VLMs) that are expensive and still inaccurate on many scripts.
By Xingsong Ye, Yongkun Du, Jiaxin Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.