跳到正文
arXiv cs.CY· Yousef Attia, Jaros{\l}aw A. Chudziak·· 9 小时前AI 评分31

语言模型能排序政党立场却无法定位其刻度

Knowing Which, Not How Much: Language Models Order Party Positions They Cannot Place

AI 导读

语言模型对政党立场的判断存在“会排序、不会定量”的分离:在日韩政党立场测试中,要求对四个政党排序时模型表现超过最强固定答案基线两倍以上,但要求把单一政党放到同一量表上时却低于该基线。

正文 · 原文

View PDF HTML (experimental)

Abstract:Language models are increasingly asked where real actors stand: what a party's platform prioritises, what an institution has committed to. Such judgments are gradable, carrying both an order and a magnitude. Nearly every benchmark that tests them asks about one entity at a time, then reads that score as the model's knowledge of the domain. Order and magnitude, however, need not be acquired together. Text about social entities carries far more comparison than measurement, so a model may hold a dependable ordering with no anchored scale beneath it. We ask whether a model can place a documented social position on a scale or only order the actors it compares, and what that means for how such knowledge is measured. We test this on party positions in Japan and South Korea, where the Comparative Manifesto Project's coded data supplies a rare external answer key. Asked to order four parties, models beat the strongest fixed-answer baseline by more than a factor of two; asked to place one party on the same scale, they fall below it. The ordering survives inside those failed placements, and supplying the coded values lifts placement to near ceiling. Comparative and absolute questions about the same fact therefore measure different things here, and for party issue salience a benchmark built from one cannot speak for the other.
Comments: Accepted at the 40th Pacific Asia Conference on Language, Information and Computation (PACLIC 40)
Subjects: Computation and Language (cs.CL); Computers and Society (cs.CY)
Cite as: arXiv:2611.21878 [cs.CL]
  (or arXiv:2611.21878v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.21878

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yousef Attia [view email]
[v1] Mon, 16 Nov 2026 20:44:26 UTC (106 KB)

来源:arXiv cs.CY · arxiv.org