Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy
Researchers added Greek to the Cosmos3 vision-language-action policy using only machine-rephrased instructions, finding bilingual training reaches roughly two fifths of English performance.
The paper studies localizing the open Cosmos3 vision-language-action robot policy to Greek without architectural changes, using machine-rephrased instructions only. Bilingual training yields a consistent 6.7-7.1 point margin over controls on a 90-task, three-seed evaluation suite, while Greek-only training gains at most 2.7 points. Several common evaluation instruments, including color-histogram metrics and single-goal benchmarks, produced false conclusions, and results were dominated by seed variation. The authors recommend building guaranteed-null baselines and replicating low-resource-language results across seeds.
- Bilingual training improves Greek robot policy performance by 6.7-7.1 points over controls.
- Multilingual text tower without Greek data stays at wrong-instruction floor performance.
- Single-run comparisons unreliable due to seed variation; three seeds per arm needed.
- Training on seven phrasings per task approximately halves translator-phrasing overfitting penalty.
From the original article
1 min read at huggingface.co
Robot foundation models are trained and evaluated predominantly in English, and robot demonstration corpora do not exist for most languages. We study the addition of Greek to an open vision-language-action stack using only machine-rephrased instructions and no architecture changes. The main challenge is measurement rather than translation.
By Ayoub Kirouane, Georgios Giaples, Christos Petrocheilos · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.