arXiv cs.CL· Liping Yang, Yongqi Leng, Jingting Zheng, Yuqi Ren, Shaolin Zhu, Deyi Xiong·· 12 小时前AI 评分38
ABench：面向分类价值观的中文大语言模型对抗性基准
ABench: Adversarily Benchmarking Large Language Models across Taxonomized Values
AI 导读
研究者提出 ABench，一个针对中文分类价值观的细粒度手工构建基准，基于 158 个细分价值子类设计 2,895 道开放式问题，通过角色扮演和逻辑陷阱探测模型在对抗性诱导下的立场稳定性。对 16 个主流 LLM 的实验显示，多数模型在复杂价值观对抗攻击下得分低于 60。该基准已被 EMNLP 2026 主会接收，数据将公开释放。
正文 · 原文
Abstract:Ensuring the alignment of Large Language Models (LLMs) with diverse cultural values is critical for their safe global deployment. However, existing alignment benchmarks are predominantly English-centric or confined to simple discriminative tasks. Moreover, they often fail to effectively evaluate model robustness against complex, adversarial inductions within specific cultural contexts. To address these limitations, we propose ABench, a fine-grained, handcrafted benchmark specifically tailored to taxonomized values in Chinese. Grounded in a rigorous hierarchical taxonomy comprising 158 derived value subcategories, ABench consists of 2,895 open-ended questions. These questions employ techniques such as role-playing and logical traps to probe the stability of models' stances in adversarially induced contexts. Furthermore, we propose a weighted automated evaluation framework that utilizes question-specific scoring criteria for nuanced assessment. Extensive experiments on 16 mainstream LLMs reveal that most models exhibit significant vulnerabilities when subjected to sophisticated adversary attacks on values (scoring below 60). This underscores the need for more robust, culturally specific alignment strategies. Our data will be publicly released to promote further research on taxonomized value alignment.
| Comments: | Accepted to EMNLP 2026 (Main Conference). 16 pages, 6 figures, 7 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.27611 [cs.CL] |
| (or arXiv:2610.27611v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27611 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Liping Yang [view email]
[v1]
Tue, 20 Oct 2026 04:23:14 UTC (835 KB)
来源：arXiv cs.CL · arxiv.org