跳到正文
arXiv cs.CL· Md Tahmid Rahman Laskar, Xue-Yong Fu, Shashi Bhushan TN·· 23 小时前AI 评分38

决策模型能否在真实呼叫中心分类与评判任务中取代 LLM?

Can Decision Models Replace LLMs in Real-World Contact Center Classification and Judging Tasks?

AI 导读

一项 arXiv 研究在零样本设置下,用 8 个呼叫中心用例(6 个分类任务、2 个评判任务)对比了 9 个决策模型与 13 个生成式 LLM。结果显示,许多决策模型在分类任务上可匹配前沿 LLM,且基于 API 的闭源决策模型单次请求成本最低;其高置信度预测更准确,可用于生产环境的选择性路由。

正文 · 原文

View PDF HTML (experimental)

Abstract:Contact center systems often need to make bounded decisions, such as identifying a caller's intent, retrieving the knowledge-base article that answers a query, assessing agent compliance, or judging the output of AI models used in production. Decision models return a typed answer and its probability directly, making them a promising cost-effective alternative to generative Large Language Models (LLMs) in such bounded conversational tasks. However, their effectiveness in real-world contact center applications remains unclear. In this paper, we compare nine decision models against thirteen generative LLMs in zero-shot settings across eight contact center use cases: six classification tasks and two judging tasks. Our experiments demonstrate that many decision models can match the performance of frontier LLMs on contact center classification, and API-based closed decision models achieve this at the lowest per-request cost among the API models we evaluate. We further observe that their confident predictions are more accurate, enabling selective routing in real-world production scenarios. Across classification tasks, the main weakness of most decision models is abstention: they rarely choose the no-answer option, which accounts for most of their gap to the strongest LLMs (although Gemini-Decision-1 is the notable exception in this regard). We also find that the judging quality of decision models drops sharply for smaller models, and they are also more prone to position bias. In short, our findings provide practical guidance for selecting decision models based on abstention behavior, confidence, and deployment cost, rather than overall accuracy alone.
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2611.22949 [cs.CL]
  (or arXiv:2611.22949v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.22949

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Md Tahmid Rahman Laskar [view email]
[v1] Tue, 17 Nov 2026 15:39:40 UTC (106 KB)

来源:arXiv cs.CL · arxiv.org