arXiv cs.AI· Massi-Nissa Abboud, Aladin Djuhera, Elena Cabrio, Holger Boche·· 12 小时前AI 评分34
POLI-BIAS：测量大语言模型在国际政治冲突中的偏见
Poli-Bias: Understanding and Measuring Large Language Model Biases in International Political Conflicts
AI 导读
POLI-BIAS 是一个反事实框架，通过系统交换国家身份来测量 LLM 是否对法律上等价的冲突场景区别对待。研究覆盖 5,600 组提示词对和七个不同模型家族与规模的开源权重 LLM，发现模型对处于实际冲突或紧张关系中的国家处理得更不均衡，对等价行为的描述、评价和辩护存在差异。该框架将响应差异分解为五个可解释维度，用于细粒度审计 LLM 的政治公正性。
正文 · 原文
Abstract:Measuring political bias in large language models (LLMs) remains challenging as it can manifest through subtle differences in framing, argumentation, and legal reasoning that are difficult to capture with a single metric. In this work, we introduce POLI-BIAS, a counterfactual framework for measuring whether LLMs treat legally equivalent conflict scenarios differently depending on the countries involved. POLI-BIAS compares responses to paired prompts in which country identities are systematically swapped across diverse geopolitical relationships, legal violations, and reasoning tasks. Rather than reducing bias to a single judgment, our framework decomposes response disparities into five interpretable dimensions, revealing how and where unequal treatment manifests. Across 5,600 prompt pairs and seven open-weight LLMs from diverse model families and sizes, we find that models treat countries in active conflict or tension less evenly than fictional or uncontested countries, describing, evaluating, and defending equivalent actions differently. Our results thus establish POLI-BIAS as a fine-grained framework for auditing political even-handedness in LLMs.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.06123 [cs.AI] |
| (or arXiv:2608.06123v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.06123 arXiv-issued DOI via DataCite |
Submission history
From: Massi-Nissa Abboud [view email]
[v1]
Thu, 6 Aug 2026 14:54:30 UTC (362 KB)
[v2]
Mon, 10 Aug 2026 14:43:02 UTC (362 KB)
[v3]
Thu, 15 Oct 2026 20:35:07 UTC (308 KB)
来源：arXiv cs.AI · arxiv.org