arXiv cs.CL· Mohammed Nayeem Khan, Sarun Kumar K S·· 10 小时前AI 评分26
PolyRoute：覆盖 12 种语言与代码混合文本的类型化决策路由基准，含基线系统与排行榜
PolyRoute: A Twelve-Language and Code-Mixed Benchmark for Typed-Decision Routing, with Baselines and a Leaderboard
AI 导读
PolyRoute 是一个覆盖 12 种语言、9 种书写系统的类型化决策路由基准，包含 322 条母语支持消息、156 条代码混合或罗马化消息（Hinglish、Spanglish、Arabizi）及 967 条可选母语训练数据，标签由 Qwen3.8-27B 生成并经盲审二遍校验，gemma4-31b 在 97.2% 的测试项上与其一致。
正文 · 原文
Abstract:Typed-decision encoders answer a question about a text in one forward pass, but their behaviour outside English is rarely measured on a fixed yardstick. We release PolyRoute: 322 native-language support messages in 12 languages and 9 writing systems, 156 code-mixed or romanised messages (Hinglish, Spanglish, Arabizi), 967 optional native training rows, and a protocol for a three-way department-routing decision. Labels come from Qwen3.8-27B with a blind second pass (91% of candidates kept); an independent model, gemma4-31b, agrees on 97.2% of test items, and the largest character 5-gram similarity between any test text and any training text is 0.33. A leaderboard of 13 systems gives 61.2% for the English-root checkpoint, 77.3% for the shipped multilingual one and 96.6% for the best fine-tune; TF-IDF with logistic regression reaches 91.9%. On four withheld languages the three multilingual systems score 77%, 90% and 93%. On code-mixed text the English root (85%) is nominally ahead of the shipped multilingual checkpoint (79%; p=0.134). The split resolves differences only above about 4 points. All texts and labels are LLM-generated.
| Comments: | 25 pages, 18 figures, 26 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13843 [cs.CL] |
| (or arXiv:2611.13843v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13843 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sarun Kumar [view email]
[v1]
Tue, 10 Nov 2026 10:04:27 UTC (1,905 KB)
来源：arXiv cs.CL · arxiv.org