arXiv cs.CL· Zihao Sun·· 11 小时前AI 评分34
从语言到国家到效价：跨 LLM 刻板印象偏见的多语言审计
From Language to Country to Valence: A Multilingual Audit of Stereotype Bias Across LLMs
AI 导读
一项多语言审计用 12,000 条英、西、法、德本地撰写的 Reddit 帖子评估 3 个开源权重模型和 3 个闭源 API 模型的性别、种族与宗教偏见，仅翻译搜索词以保留本地语境。宗教在所有模型-语言组合中绝对选择失衡最高，西班牙语在性别上最高，德语在种族和宗教上最高，且 API 模型整体得分低于开源权重模型。英文性别条目中负面项多于正面项，澳大利亚与墨西哥语境高于加拿大与西班牙对应语境。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly considered for education, recruitment, workplace decision support, and other consequential applications, making pre-deployment social-bias detection essential. Yet major benchmarks are English-centred or translated, focus mainly on negative stereotypes, and may miss linguistic and cultural variation. We investigate gender, racial, and religious bias using 12,000 locally written Reddit posts in English, Spanish, French, and German. Only search terms are translated, preserving local context. The dataset is balanced by language and category; English gender is additionally balanced by stereotype valence. Three open-weight and three proprietary API-based models are evaluated with a two-blank design that reduces grammatical-gender cues. Religion has the highest absolute choice imbalance in every model-language condition. Spanish is highest for gender, whereas German is highest for race and religion. The evaluated API models score below the open-weight models throughout. Negative English-gender items exceed positive items; Australian and Mexican contexts exceed their Canadian and Spanish counterparts. These descriptive differences measure choice concentration, not stereotype endorsement or causal country effects. The crossed patterns show how aggregate scores can conceal category- and context-specific behaviour, motivating closer inspection before deployment.
| Comments: | 10 pages. Author preprint; conference review ongoing |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.02585 [cs.CL] |
| (or arXiv:2611.02585v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02585 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zihao Sun [view email]
[v1]
Mon, 2 Nov 2026 06:11:09 UTC (38 KB)
来源：arXiv cs.CL · arxiv.org