arXiv cs.CL· Evelyn Y. Garland, Carola F. Berger·· 10 小时前AI 评分20
1020 Group 的 Pterodactyl 系统：面向 WMT26 翻译质量评估的高效低成本方案
Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation
AI 导读
1020 Group 为 WMT26 自动质量评估共享任务提交了 Pterodactyl 系统，专攻子任务 2 和 3，即片段级质量分数预测与无错误片段检测。该系统基于 LLM-as-a-judge 方法，通过 ROC 分析指导提示词工程与模型选择，定位为高度可定制、超低成本的二分类方案，用于翻译分流与数据筛选。团队还针对子任务 3 开发了多个适配变体。
正文 · 原文
Abstract:In this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.
| Comments: | 6 pages, 4 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.14480 [cs.CL] |
| (or arXiv:2611.14480v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14480 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Proceedings of the Eleventh Conference on Machine Translation (WMT 2026), pp. 1638-1643, 2026 |
| Related DOI: | https://doi.org/10.18653/2026.wmt-main.96
DOI(s) linking to related resources |
Submission history
From: Evelyn Garland [view email]
[v1]
Tue, 10 Nov 2026 18:41:37 UTC (657 KB)
来源：arXiv cs.CL · arxiv.org