arXiv cs.CY· Haonan Huang·· 11 小时前AI 评分36
拆解 LLM 道德判断中的是/否偏差：立场、位置、词与裁决
Decomposing Yes-No Bias in LLM Moral Judgment: Stance, Position, Word, and Verdict
AI 导读
研究提出交叉心理测量电池，将问题极性、印刷顺序与答案标签扩展为 2x2 和 2x2x2 交叉设计，以分离 LLM 是/否回答中的立场、位置、词与裁决效应。在 20 个道德困境、Claude/GPT/Gemini 十个前沿模型 23 种推理配置共 135 万次试验中，表面是/否偏差几乎纯属词与位置等表层因素，而部分最新模型翻转对比接近零却隐藏高达 +0.42 的裁决依附。
正文 · 原文
Abstract:Large language models (LLMs) increasingly issue binary verdicts - safety gates, judge decisions, value probes - and the answer is read as the model's judgment. A settled judgment should be coherent across paired phrasings of the same question: yes to "do you approve?" should mirror no to "do you oppose?". LLMs often violate this coherence, and in a yes/no question the verdict, the answer word, and the printed position coincide, so a conventional flipped-question audit yields one contrast that cannot say what moved. We introduce a crossed psychometric battery that extends the flip to full 2x2 and 2x2x2 crossings of question polarity, printed order, and answer label, separating stance from position, word, label, and verdict-role effects; an independent graded instrument estimates each model's stance $\theta$ with no binary format; and a two-parameter fit (framing susceptibility $m$, decisiveness $s$) separates robustness from saturation that hides susceptibility. Applying the battery to twenty moral dilemmas - 1.35 million trials over ten frontier models from the Claude, GPT, and Gemini families in 23 reasoning configurations - we find response-form bias widespread but never uniform. The largest apparent yes-no biases are almost purely surface (word and position), while several of the newest models show the reverse: near-zero flip contrasts concealing verdict attachment up to +0.42, a component no surface counterbalance can expose, replicated across label alphabets. Requested reasoning reshapes these channels rather than removing them. When subjects state decisions in free text and a separate model transcribes them, two judge families agree on 99.3% of jointly scored transcriptions with zero opposing committed labels, and the transcribed stance converges with the graded ruler (median $r=0.87$): response form is part of the instrument, and the judgment can be measured apart from it.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2607.05552 [cs.CL] |
| (or arXiv:2607.05552v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05552 arXiv-issued DOI via DataCite |
Submission history
From: Haonan Huang [view email]
[v1]
Mon, 6 Jul 2026 18:37:43 UTC (2,243 KB)
[v2]
Fri, 6 Nov 2026 03:52:26 UTC (1,872 KB)
来源：arXiv cs.CY · arxiv.org