arXiv cs.CL· Md Sabbir Hossen, Anichur Rahman, Pabon Shaha, Airin Afroj Aishi, Andrew H. Sung, Md Shohel Rana·· 12 小时前AI 评分31
高效推理谁来买单？基于置信度的小模型到大模型 LLM 路由在任务与语言间的表现
Who Pays for Efficient Inference? Confidence-Based Small-to-Large LLM Routing Across Tasks and Languages
AI 导读
一项系统评估在三个模型家族、四种语言和三类任务上测试了基于置信度的小模型到大模型路由，覆盖 72 种配置、42,288 条预测。
正文 · 原文
Abstract:Small-to-large language model routing can reduce inference cost by escalating difficult inputs while retaining easier ones on a smaller model. Its success, however, depends on identifying examples that benefit from escalation, and the reliability of confidence across multilingual tasks remains unclear. We present a systematic evaluation of confidence-based routing across three model families, four languages, and three task types, covering 72 model, dataset, language, and scale configurations and 42,288 predictions. We compare global confidence routing with random routing, equal-quota confidence routing, and an oracle upper bound. Confidence identifies beneficial escalations on the Belebele reading comprehension and SIB-200 topic classification datasets, with mean correction-detection AUCs of 0.612 and 0.705. At 50% escalation, it improves over random routing by 1.40--4.67 percentage points on Belebele and 2.54--5.63 points on SIB-200. In contrast, the mean AUC on the MMLU-ProX-Lite knowledge and reasoning dataset is only 0.510, and routing provides no consistent improvement over random selection because small models often remain highly confident while incorrect. Global confidence routing also creates language escalation gaps up to 30.33 percentage points, whereas equal-quota routing removes this disparity with little aggregate accuracy cost. These results show that confidence routing is task-dependent and can allocate computation unevenly across languages. Code is available at this https URL
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23249 [cs.CL] |
| (or arXiv:2610.23249v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23249 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Md. Sabbir Hossen [view email]
[v1]
Fri, 16 Oct 2026 10:23:41 UTC (132 KB)
来源：arXiv cs.CL · arxiv.org