arXiv cs.CL· P N Bhargav Teja, Velpula Lokesh Reddy, Madhu Basumata, Susen Sarkar, Rajasekhara Reddy Duvvuru Muni·· 6 天前AI 评分20
SmartSalesBench：面向 LLM 销售智能体的抗压多轮对话基准
SmartSalesBench: A Resistance-Aware Multi-Turn Benchmark for Evaluating LLM Sales Agents
AI 导读
SmartSalesBench 是首个针对 LLM 销售智能体的多轮对话基准，让智能体面对具有心理依据、会抵抗和质疑的买家角色，对话最长 20 轮。基准包含 5 个智能体画像、12 个买家角色、覆盖 6 个价格档的 15 款手机产品，以及 4 组对照实验共 13,680 场会话，销售方与买家角色生成均基于 RAG。
正文 · 原文
Abstract:We present SmartSalesBench, the first multi-turn conversational benchmark explicitly designed to evaluate large language model (LLM) sales agents against psychologically grounded, resistant buyer personas. Existing task-oriented dialogue benchmarks assume cooperative users and measure task completion; SmartSalesBench inverts this assumption, placing agents against personas programmed to resist and challenge across conversations of up to twenty turns. The benchmark includes five agent profiles, twelve buyer personas, fifteen smartphone products across six price tiers, and four controlled experiments with 3 iterations totaling 13,680 sessions. Both sales-agent and buyer persona generations are grounded through retrieval-augmented generation (RAG). We introduce nine evaluation metrics grouped into three tiers: primary outcome metrics (CR, BRS, RIR), secondary quality metrics (SMS, PFS, ETI), and exploratory metrics planned for future reporting (ITS, ARI, SPI). Our evaluation combines automated LLM-as-judge scoring with internal human assessment (n = 50, \k{appa} = 0.71, Pearson r = 0.81 with BRS).
| Comments: | Request for review |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12975 [cs.AI] |
| (or arXiv:2610.12975v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12975 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Susen Sarkar [view email]
[v1]
Fri, 9 Oct 2026 04:21:12 UTC (2,962 KB)
来源：arXiv cs.CL · arxiv.org