跳到正文
arXiv cs.AI· Yezhou Cheng, Zehua Yang, Bojun Lin·· 10 小时前AI 评分24

基于带符号词汇证据的风险校准意图路由,提升对话 AI 任务分流可靠性

Risk-Calibrated Intent Routing for Conversational AI via Signed Lexical Evidence

AI 导读

研究者提出一种双特征置信度模型,将分类器最高两个分数的差值与带符号词汇证据结合,用于对话 AI 的意图路由。在 BANKING77、CLINC150、HWU64 上,风险-覆盖率曲线下面积分别降低 15.8%、15.1%、11.8%;5% 错误目标下,BANKING77 和 HWU64 的接受请求比例分别提升 1.83 和 5.14 个百分点。

正文 · 原文

View PDF HTML (experimental)

Abstract:Conversational AI has become widely used as advances in language models let people find information, get writing help, and ask for practical guidance in their own words. When an assistant handles specific tasks, it must infer what the user wants and decide whether to act on that prediction. A sentence classifier's confidence can miss clues in the wording. We combine the gap between the classifier's two highest scores with signed lexical evidence, a score based on words and short character patterns. The score is positive when those patterns favor the predicted intent and negative when they favor another. We use separate calibration data and a statistical upper bound on the error rate to set the minimum confidence required to accept a request. Across ten runs per dataset, our method reduces the area under the risk-coverage curve, which summarizes error as more requests are accepted, by 15.8%, 15.1%, and 11.8% on BANKING77, CLINC150, and HWU64, respectively, compared with a confidence model that uses only sentence classifier scores. At a 5% error target, the share of accepted requests rises by 1.83 percentage points on BANKING77 and 5.14 points on HWU64. CLINC150 already has near-full coverage at that target. At a 2% target, calibration finds a rule that accepts some requests in all 30 runs. Our contribution is a two-feature confidence model that improves routing by measuring how strongly the wording supports or challenges the original intent prediction.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2610.00262 [cs.CL]
  (or arXiv:2610.00262v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2610.00262

arXiv-issued DOI via DataCite

Submission history

From: Yezhou Cheng [view email]
[v1] Thu, 24 Sep 2026 07:06:22 UTC (142 KB)
[v2] Wed, 18 Nov 2026 03:17:05 UTC (145 KB)

来源:arXiv cs.AI · arxiv.org