arXiv cs.CY· Nathanael Jo, Nikhil Garg, Manish Raghavan·· 12 小时前AI 评分34
研究质疑 LLM 同质化结论：同质化测量本质上是主观的
The Subjectivity of Monoculture
AI 导读
两项实验对 LLM 同质化的既有发现提出实质异议：在其中一个数据集上，纳入题目难度后同质化几乎消失。作者据此形式化定义了同质化的测量方式，指出其依赖两个关键选择——"独立"所对应的基线零模型，以及所考察的模型与题目总体，因此同质化本质上是主观的。论文已被 NeurIPS 2026 接收，并为后续评测者给出具体准则。
正文
Abstract:Machine learning models---including large language models (LLMs)---are often said to exhibit monoculture, where outputs agree strikingly often. However, we present two experiments that substantially disagree with prior findings of monoculture. In particular, we find that monoculture virtually disappears in one dataset when we include item difficulty compared to previous works that do not.
Informed by these findings, we theoretically formalize what it means to measure monoculture. We show that monoculture is inherently subjective, relying on two key decisions: 1) the baseline null model for what "independence" should look like; and 2) the population of models and items under consideration. We conclude with concrete guidelines for future evaluators to ensure the robustness of their results.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Computers and Society (cs.CY); Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.24086 [cs.CY] |
| (or arXiv:2602.24086v2 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2602.24086 arXiv-issued DOI via DataCite |
Submission history
From: Nathanael Jo [view email]
[v1]
Fri, 27 Feb 2026 15:23:14 UTC (804 KB)
[v2]
Fri, 23 Oct 2026 18:16:06 UTC (1,068 KB)
来源：arXiv cs.CY · arxiv.org