跳到正文
arXiv cs.CY· Sangho Kim, Heejin Kim, Hyunggeun Jeon, Yoonhee Park, Jaejin Lee·· 23 小时前AI 评分35

Polar:评估 LLM 政治偏见的基准

Polar: A Benchmark for Evaluating Political Bias in LLMs

AI 导读

研究者推出 Polar,一个含 4,026 道多选题的基准,通过选项级似然而非提示词生成来测量 LLM 政治偏见,覆盖两个意识形态轴与 Manifesto Project 的八个议题类别,并在美国与韩国政治语境下平行评测。

正文 · 原文

View PDF HTML (experimental)

Abstract:Political bias in large language models (LLMs) is increasingly significant, but difficult to measure reproducibly across political and linguistic contexts. We introduce Polar, a 4,026-instance multiple-choice benchmark that measures political bias through option-level likelihoods rather than prompt-based generation. Polar covers two ideological axes and eight issue categories derived from the Manifesto Project, and evaluates models in parallel across U.S. and South Korean political contexts. Across 38 LLMs, measured bias varies systematically with political context, issue category, model group, and presentation language. All models lean left-progressive on U.S. political content, but show more centered and mixed patterns on South Korean content. Translation experiments further show that presentation language alone can shift measured bias. These findings highlight the need for multilingual and cross-contextual evaluation of political bias in LLMs.
The code and dataset are publicly available on GitHub (this https URL) and Hugging Face (this https URL).
Comments: Accepted to Findings of AACL-IJCNLP 2026 Code and Dataset: this https URL ; this https URL
Subjects: Computation and Language (cs.CL); Computers and Society (cs.CY)
Cite as: arXiv:2606.12922 [cs.CL]
  (or arXiv:2606.12922v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2606.12922

arXiv-issued DOI via DataCite

Journal reference: Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Related DOI: https://doi.org/10.18653/2026.findings-aacl.181

DOI(s) linking to related resources

Submission history

From: Sangho Kim [view email]
[v1] Thu, 11 Jun 2026 05:26:28 UTC (822 KB)
[v2] Wed, 18 Nov 2026 07:56:26 UTC (887 KB)

来源:arXiv cs.CY · arxiv.org