arXiv cs.CY· Eduardo Levy Yeyati, C\'esar Ciappa, Milagros Onofri·· 11 小时前AI 评分42
研究：LLM 意识形态评估存在语言非不变性，GPT-3.5、GPT-4o、GPT-5.2、Qwen-2.5、Mistral Large 跨语言排名会反转
Language Non-Invariance in the Ideological Evaluation of Large Language Models
AI 导读
一项基于 Political Compass 的研究发现，GPT-3.5、GPT-4o、GPT-5.2、Qwen-2.5 和 Mistral Large 五个 LLM 的经济与社会轴意识形态位置并非语言不变：相对英语，西班牙语或法语下十个模型两两排名中有四到五个发生反转。
正文 · 原文
Abstract:Recent multilingual studies show that political evaluations of large language models (LLMs) can vary across languages. This paper examines how robust this non-invariance is to questionnaire translation, surface wording, stochastic model variation, and instrument choice, and how much it matters for cross-model comparison. Using the Political Compass, the analysis compares the positions of five LLMs, GPT-3.5, GPT-4o, GPT-5.2, Qwen-2.5, and Mistral Large, on its economic and social axes in English, Spanish, and French, with repeated stochastic draws. Additional exercises use independent Spanish and French translations, paraphrased versions in all three languages, and an alternative ideological instrument. Measured ideological positions are not language-invariant: relative to English, between four and five of the ten pairwise model rankings reverse in Spanish or French, depending on the language and axis. The broad cross-language pattern persists under independent translations, indicating that it is not solely an artifact of the official questionnaire versions. Yet the magnitude of particular shifts, and in some cases their sign, remains sensitive to wording and instrument choice. The evidence therefore supports a measurement interpretation: ideological audits recover language- and instrument-conditioned outcomes rather than a language-free political attribute. Because a single provider serves distinct linguistic constituencies through a common product, without users observing what other constituencies receive, this lack of comparability also bears on political accountability and cross-model comparisons.
| Subjects: | Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.41135 [cs.CY] |
| (or arXiv:2610.41135v1 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41135 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Eduardo Levy Yeyati [view email]
[v1]
Fri, 30 Oct 2026 01:43:37 UTC (498 KB)
来源：arXiv cs.CY · arxiv.org