arXiv cs.CL· Tian Lan, Han Zhang, Shanshan Wang, Yahan Yu, Jiang Li, Yugo Murawaki·· 2 天前AI 评分24
APRiL：面向大语言模型中文社交意识的开源基准
APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models
AI 导读
研究者推出 APRiL，一个面向中文社交意识的开源基准，包含 2,196 个社交意识评估实例，覆盖六类场景与社交场景分析、社交意识回应两项任务。在来自六个模型家族的 12 个模型上测试发现，模型普遍存在"理解到回应"的落差，高质量场景分析能提升回应质量，而弱分析可能拉低回应质量。数据与代码已公开。
正文 · 原文
Abstract:Large language models are increasingly used in social interactions, where appropriate responses require recognizing implicit cues, inferring mental states, identifying conflicts, and considering potential consequences. However, existing benchmarks often focus on specific social abilities, constrained task formats, or particular interaction settings, offering limited coverage of social awareness across diverse contexts. To address this gap, we introduce APRiL, an open-ended benchmark for Chinese social awareness with instance-specific rubrics. APRiL contains 2,196 Social Awareness Evaluation Instances across six categories and two complementary tasks: Social Scenario Analysis evaluates models' understanding of social cues, mental states, and conflict structures, while Socially Aware Response assesses whether they can translate this understanding into appropriate, natural responses. Experiments on 12 models from six families reveal a consistent understanding-to-response gap. Higher-quality scenario analyses improve responses, while weak analyses may reduce response quality. APRiL provides a systematic resource for advancing socially aware language models in Chinese contexts. Our data and code are available at this https URL.
| Comments: | 26 pages, 9 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20785 [cs.CL] |
| (or arXiv:2610.20785v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20785 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tian Lan [view email]
[v1]
Wed, 14 Oct 2026 18:28:42 UTC (1,408 KB)
来源：arXiv cs.CL · arxiv.org