Pterodactyl targets low-cost automated translation quality scoring
Original title: Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation
1020 Group's WMT26 system uses an LLM judge for segment scores and error-free segment detection.
1020 Group submitted Pterodactyl to the WMT26 shared task on automated quality evaluation. The system targets subtasks 2 and 3: segment-level quality score prediction and detection of error-free segments. It uses an LLM-as-a-judge design, with prompt engineering and model selection guided by ROC analysis, and is described as customizable and ultra-low cost for translation triage and data curation. Variants were adapted specifically for subtask 3.
- The submission covers WMT26 subtasks 2 and 3.
- Prompt and model choices were guided by ROC analysis.
- Additional variants were adapted for error-free segment detection.
- Products
- Pterodactyl
- Organizations
- 1020 Group
From the original article
1 min read at arxiv.orgIn this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation.
By Evelyn Y. Garland · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.