arXiv cs.CL· Ali Zamani, Soroush Esfahanian, Erfan Shahabi, Mohammad Taher Pilehvar, Azadeh Shakery, Yadollah Yaghoobzadeh·· 3 天前AI 评分30
PerSo：在真实说服场景中考察 LLM 社会智能的多个维度
PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
AI 导读
研究者提出 PerSo 基准，基于 Reddit ChangeMyView 社区的真实说服对话，用 LLM-in-the-loop 流程构建评估任务并经人工标注验证，同时考察 LLM 的认知智能与行为智能。对多个 LLM 的评测显示，性能总体随模型规模提升但各维度表现不一，最强 LLM 在行为任务上略超最佳人类表现，而最佳人类参与者在认知任务上仍保持领先。
正文 · 原文
Abstract:Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. Persuasion provides a natural testbed in which both aspects are simultaneously at play, yet prior benchmarks typically evaluate them in isolation using synthetic scenarios. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an LLM-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse LLMs shows that performance generally scales with model size but varies across dimensions. The strongest LLM slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.
| Comments: | 25 pages, 17 figures, 10 tables. Ali Zamani and Soroush Esfahanian contributed equally |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.20548 [cs.CL] |
| (or arXiv:2610.20548v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20548 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Erfan Shahabi [view email]
[v1]
Wed, 14 Oct 2026 17:39:35 UTC (5,138 KB)
来源：arXiv cs.CL · arxiv.org