Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy
Researchers added Greek to the Cosmos3 vision-language-action policy using only machine-rephrased instructions, finding bilingual training reaches roughly two fifths of English performance.
The paper studies localizing the open Cosmos3 vision-language-action robot policy to Greek without architectural changes, using machine-rephrased instructions only. Bilingual training yields a consistent 6.7-7.1 point margin over controls on a 90-task, three-seed evaluation suite, while Greek-only training gains at most 2.7 points. Several common evaluation instruments, including color-histogram metrics and single-goal benchmarks, produced false conclusions, and results were dominated by seed variation. The authors recommend building guaranteed-null baselines and replicating low-resource-language results across seeds.