arXiv cs.CL· Tzu-Ling Lin, Dong-Ting Yao, Teng-Fang Hsiao, Wei-Chih Chen, Hong-Han Shuai·· 10 天前AI 评分25
HalluPeer：面向科学同行评审幻觉检测的分类学驱动基准
HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews
AI 导读
HalluPeer 是一个用于检测科学同行评审中幻觉的基准，提供论文内容、人工评审与注入幻觉评审对齐的三元组，支持检测、分类与定位。该基准基于同行评审专属幻觉分类学构建，在 12K 篇论文和 38K 条评审上实验显示，现有检测器难以区分幻觉与合理批评，且这些幻觉模式在真实评审中确实存在。该工作已被 EMNLP Findings 2026 接收。
正文 · 原文
Abstract:The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability. Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers. We introduce HalluPeer, a benchmark for detecting hallucinations in scientific peer reviews, providing aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Our pipeline induces a peer-review-specific hallucination taxonomy, identifies review contexts, and injects hallucinations with automated filtering. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, while evaluation on authentic reviews demonstrates that HalluPeer-defined hallucination patterns occur in real peer reviews, highlighting the critical need for source-aware verification. Our project page can be found in this https URL
| Comments: | Accepted to EMNLP Findings 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.03580 [cs.AI] |
| (or arXiv:2609.03580v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.03580 arXiv-issued DOI via DataCite |
Submission history
From: Tzu-Ling Lin [view email]
[v1]
Thu, 3 Sep 2026 09:27:54 UTC (5,226 KB)
[v2]
Wed, 7 Oct 2026 09:42:16 UTC (5,112 KB)
来源：arXiv cs.CL · arxiv.org