arXiv cs.CL· Duaa Nakshbandi, Zhiyu An, Yiping Li, Sachin Giri, Wan Du·· 2 天前AI 评分34
LLM 能否表示复数谓述中的真值间隙？HomogeneityNLI 基准揭示 Bivalence Bias
Neither True nor False: Do LLMs Represent Truth-Value Gaps in Plural Predication?
AI 导读
研究者构建 HomogeneityNLI 诊断数据集（1,000 个最小对，覆盖 10 种逻辑条件），评测 7 个开放权重模型，发现普遍存在 Bivalence Bias：零样本下无模型能在超过 15% 的场景中同时识别两种极性的真值间隙，否定式复数定指在 84% 的情况下被判定为 False。
正文 · 原文
Abstract:Large Language Models (LLMs) are increasingly being used to summarize and verify claims in critical sources that inform consequential decisions. Such claims are full of plural definites. If three of five patients improved after taking drug A, the claim the patients improved after drug A is not true, since two did not, yet its negation, the patients didn't improve after drug A, is not true either, since three did. In formal semantics, this truth-value gap is known as the Homogeneity Gap: both the plural definite and its negation are neither true nor false when the property holds for only a fraction of the plural referent. A model that cannot represent this gap turns mixed evidence into a clean verdict, an error that can be costly in high-stakes use. We introduce HomogeneityNLI, a diagnostic dataset of 1,000 minimal pair items crossing determiner, polarity, and group composition across ten logical conditions. Evaluating seven open-weight models, we uncover a pervasive Bivalence Bias: models assign plural definites their homogeneous truth conditions. In the zero-shot setting, no model recognizes the gap for both polarities in more than 15% of scenarios, and negated definites are judged False in 84% of cases. Explicit linguistic rules, chain-of-thought, and enumeration prompts produce unstable, model-specific effects that often entrench the bias. Our findings indicate that current open-weight LLMs acquire the truth conditions of plural predication but not its trivalent logic, and cannot yet be trusted to summarize or verify claims whose evidence is mixed.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.14438 [cs.CL] |
| (or arXiv:2610.14438v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14438 arXiv-issued DOI via DataCite |
Submission history
From: Duaa Nakshbandi [view email]
[v1]
Sat, 10 Oct 2026 01:12:08 UTC (117 KB)
[v2]
Wed, 14 Oct 2026 20:31:01 UTC (118 KB)
来源：arXiv cs.CL · arxiv.org