arXiv cs.CL· Ruchira Dhar, Qiwei Peng, Anders S{\o}gaard·· 3 天前AI 评分22
语言模型评估方法论面临哪些挑战：一项跨越四十年的综述
On Challenges in Evaluation Methodology: A Review
AI 导读
一项跨越四十年的范围综述梳理出评估方法论中 22 类反复出现的关切，并构建了分类体系。研究发现，部分关切在文献中已变得不再显眼，且许多关切未被机构检查清单明确覆盖。该工作为研究社区重新审视评估方法与开展领域级干预提供了目标。
正文 · 原文
Abstract:Recent advances in language models have prompted a growing body of work that questions prevailing evaluation practices. However, many such critiques have already been discussed in natural language processing (NLP), a field with a long history of methodological reflection on evaluation. We conduct a scoping review spanning four decades and develop a taxonomy of 22 recurring concerns. For each concern, we examine how recently it was discussed in the reviewed literature and whether it receives explicit attention in institutional checklists, finding that some concerns have become less visible in literature and many lack coverage in these checklists. Our work provides an overview of the major concerns in evaluation methodology that have shaped the field and identifies potential targets for renewed investigation and field-level intervention by the research community.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.25923 [cs.CL] |
| (or arXiv:2604.25923v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.25923 arXiv-issued DOI via DataCite |
Submission history
From: Ruchira Dhar [view email]
[v1]
Wed, 1 Apr 2026 10:29:43 UTC (911 KB)
[v2]
Wed, 14 Oct 2026 06:43:26 UTC (306 KB)
来源：arXiv cs.CL · arxiv.org