arXiv cs.AI· Vasileios Saketos, William Reveillard, Alexandre Proutiere, Sebastian Dalleiger, Ming Xiao·· 3 天前AI 评分24
该延迟还是不该延迟？LLM 与拥塞专家协作的在线学习
To Defer or Not to Defer? Online Learning with LLMs and Congested Experts
AI 导读
针对 LLM 与容量受限人类专家协作的场景，研究者提出在线学习算法 KL-PBD，利用未知可靠性参数的 KL 下置信界指导延迟决策，在 K 个置信水平与时间跨度 T 下实现 O(K log T) 的间隔相关遗憾和 Õ(√KT) 最坏情况遗憾，并证明其最优性。该算法在 GSM8K、NuminaMath、MedMCQA 数据集上使用 Qwen LLM 完成验证。
正文 · 原文
Abstract:Modern AI systems increasingly combine large language models (LLMs) with human experts, using the LLM to handle routine queries while deferring uncertain cases for expert review. When expert capacity is limited, however, deferral decisions must account not only for the reliability of the LLM but also for congestion in the expert queue. We formulate this as an online sequential decision-making problem in which, for each query, the system observes the confidence signal of the LLM's answer and must decide whether to output the answer or defer the query to a capacity-constrained expert. The reliability of the confidence signal is initially unknown and must be learned from expert feedback, which is both selectively observed (only deferred queries are labeled) and delayed by queuing. We propose \texttt{KL-PBD}, a learning algorithm that uses Kullback--Leibler lower confidence bounds on the unknown reliability parameters to guide deferral decisions. For $K$ confidence levels and horizon $T$, \texttt{KL-PBD} achieves $O(K\log T)$ gap-dependent regret when the accuracy at every confidence level is bounded away from the deferral boundary, and $\widetilde{O}(\sqrt{KT})$ worst-case regret. We establish matching lower bounds, showing that these rates are optimal up to logarithmic factors. Finally, we validate our approach on synthetic instances and real-world question-answering datasets, namely GSM8K, NuminaMath, and MedMCQA, using Qwen LLMs.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15128 [cs.LG] |
| (or arXiv:2610.15128v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15128 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vasileios Saketos [view email]
[v1]
Sat, 10 Oct 2026 19:14:51 UTC (1,564 KB)
来源：arXiv cs.AI · arxiv.org