arXiv cs.CL· Maximilian Splieth\"over, Jannes L\"otfering, Dominik Bachmann, Alexander Espig, Willem Zuidema, Katrin Schulz, Henning Wachsmuth·· 12 小时前AI 评分27
文化（不）胜任力：LLM 的文化特定刻板印象检测能力研究
Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs
AI 导读
一项针对美国、荷兰和中国文化语境的研究发现，LLM 无法可靠地考虑文化语境，且会过度预测刻板印象的存在。研究系统考察了提示词语言、指令微调语言和微调数据集规模三个因素，结果显示非英语提示词能改善模型对文化语境的考量，额外指令微调可降低假阳性率，而训练数据规模对检测效果的影响呈非单调关系。该成果已被 EMNLP 2026 Findings 接收。
正文 · 原文
Abstract:Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires cultural competence. When using large language models (LLMs) for stereotype-related tasks, such as their detection, it is therefore crucial that models can access relevant cultural knowledge and adjust responses accordingly. Since LLMs have shown disparities in what cultural knowledge is available to them, their capacity regarding stereotypes may also be disparate across languages and cultural contexts (e.g., for non-English languages or for non-US contexts). In this paper, we systematically investigate the effectiveness of LLMs for culture-specific stereotype detection across three factors: (1) the prompting language, (2) the instruction tuning language, and (3) the fine-tuning dataset size. We conduct experiments for US, Dutch, and Chinese cultural contexts. Our results suggest that LLMs do not reliably account for cultural contexts and overpredict the presence of stereotypes. Non-English prompts seem to improve consideration of cultural context and thus lead to more effective predictions. Moreover, we find that additional instruction tuning reduces false-positive rates and that training data size moderates detection effectiveness non-monotonically.
| Comments: | Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22545 [cs.CL] |
| (or arXiv:2610.22545v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22545 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Maximilian Spliethöver [view email]
[v1]
Thu, 15 Oct 2026 18:42:59 UTC (2,922 KB)
来源：arXiv cs.CL · arxiv.org