arXiv cs.CL· Haonan Huang, Yuwei Lu, Chengkun Cai·· 12 小时前AI 评分31
同一判断，不同“是”：测量语言模型的多语言判断画像
Same Judgment, Different Yes: Measuring Multilingual Judgment Profiles in Language Models
AI 导读
研究从八个模型、六种语言收集 110 万条回复，在最多 105 个道德情境题上改变提问形式、答案约束与读取方式（token 或句子），考察不同提问与读取方式何时还原同一判断。
正文 · 原文
Abstract:Multilingual evaluations often score a model's opening yes or no, a token that also carries model and language answering conventions. We ask when different ways of eliciting and reading an answer recover the same judgment, and when they measure an answer habit. We collect 1.1 million replies from eight models in six languages on up to 105 moral vignettes, varying the question's form, the answer constraint, and the reading of the reply, token or sentence, in both wordings (the FAR-6 battery). Story-level judgment profiles converge across the tested readouts and question forms, except for the yes/no token after a negative interrogative read under a preset mapping: on the same replies, token and sentence profiles agree at a median r = 0.67 across cells, while sentence profiles agree with free-sentence profiles at 0.94, a known linguistic variability. With each language's story consensus removed, a model's profile is nearest to its own in another language for 39 of 48 model-language cells under proposition labels and 15 under free sentences, against 5 expected by chance. Particle, position, and named-term preferences follow their own patterns. A model can keep its judgment and change its yes: scoring that token requires establishing which proposition it affirms.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28066 [cs.CL] |
| (or arXiv:2610.28066v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28066 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuwei Lu [view email]
[v1]
Tue, 20 Oct 2026 11:50:08 UTC (14,677 KB)
来源：arXiv cs.CL · arxiv.org