arXiv cs.CL· Franco Corti, Sajid Siraj·· 11 小时前AI 评分31
用 LLM 评审团给 2024 欧盟选举宣言的环境立场打分效果如何？
How Green Is Your Manifesto? Scoring Environmental Positions in the 2024 EU Elections with an LLM Panel
AI 导读
研究用三家独立厂商的 LLM 组成评审团，对 2024 欧洲议会选举中奥地利、法国、德国、意大利、西班牙及欧洲党团的 41 份宣言逐句打分，区分环境保护与气候变化两个维度上的立场和议题显著性。
正文 · 原文
Abstract:Locating political parties on environmental and climate policy is central to the study of party competition, yet the established instruments, expert surveys and the hand coding of manifestos, are costly, periodic, and slow to extend to new documents. We ask whether a panel of large language models can serve as a cheaper and reproducible alternative. Treating each manifesto as a set of sentences, three models from independent providers score every sentence on two dimensions, environmental protection and climate change, separating a party's position on an issue from the salience it gives it, and the judgments are averaged to the manifesto level. Agreement across the models is retained as a built-in reliability signal. We apply the panel to 41 manifestos from the 2024 European Parliament election, covering Austria, France, Germany, Italy, Spain, and the European party families, and validate it against the 2024 Chapel Hill Expert Survey for the 25 parties with a like-for-like expert benchmark, using the other manifestos as diagnostic cases. The panel recovers the expert ordering of parties with strong rank agreement on both dimensions, while reading them as systematically greener than experts do, a gap we interpret as the distance between manifesto promise and governing record. Inter-model disagreement modestly flags where the instrument errs, and position and salience prove related for environmental protection but independent for climate change. The result is a transparent, low-cost, and multilingual instrument that complements rather than replaces expert and hand-coded approaches; the prompt, code, and manifesto-level results are released openly.
| Subjects: | Computation and Language (cs.CL); Computers and Society (cs.CY); General Economics (econ.GN) |
| Cite as: | arXiv:2611.02033 [cs.CL] |
| (or arXiv:2611.02033v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02033 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sajid Siraj [view email]
[v1]
Sun, 1 Nov 2026 16:27:18 UTC (83 KB)
来源：arXiv cs.CL · arxiv.org