arXiv cs.CL· Miri Liu, Xiayu Zhao, Encheng Xie, ChengXiang Zhai·· 11 小时前AI 评分34
AI 科学家能识别真正的新意吗？ClaimNov 提出 claim 级新颖性检测框架
Do AI Scientists Know What's New?
AI 导读
研究团队构建了含 1060 份提案、覆盖 proposal 与 claim 两级时间基准的新颖性检测评估框架，发现现有检测器在拒绝非新颖工作时误报率同样偏高。为此提出细粒度可解释模块 ClaimNov，以 1.5% 误报率捕获 97% 的伪装论文，并将 LLM 评审误报从 11.8% 降至 1.5%。在含拼接先验贡献的 100 份新颖提案中保留 91 份，优于最强现有方法的 76 份。
正文 · 原文
Abstract:A major challenge facing AI scientific discovery pipelines is how to determine when a generated idea is meaningfully novel. Ideally, novelty checks in these pipelines must be able to reject non-novel ideas while not rejecting too many novel ideas, and they should also be able to distinguish which parts of a proposal are non-novel, as opposed to opaquely discarding entire proposals. We design an analytical framework for evaluation of novelty checks with temporal ground truth at both the proposal- and claim-level, using four datasets of 1,060 proposals built from published papers, reviewer disputes, and generated ideas matching ones the field went on to publish. Auditing existing novelty checks with this framework, we find that checks with high rejection rates on non-novel work also tend to have high false alarms on ideas that are later realized by human researchers, and observe that checks built to evaluate proposals as a whole can point to claim-level prior art only by rejecting the whole proposal. We therefore introduce ClaimNov, a fine-grained, interpretable claim-level novelty module. ClaimNov catches 97% of disguised papers at a 1.5% false-alarm rate, and its claim-level novelty profile cuts the LLM judge's false alarms from 11.8% to 1.5%. Because its verdict rests on claim-level evidence, ClaimNov also keeps proposals that build on established contributions, rejecting only 3% of ICLR-accepted proposals with a reviewer-disputed contribution and keeping 91 of 100 novel proposals with one spliced prior contribution, against 76 for the strongest existing check. Code and data are available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36511 [cs.CL] |
| (or arXiv:2610.36511v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36511 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Miri Liu [view email]
[v1]
Tue, 27 Oct 2026 02:16:41 UTC (894 KB)
来源：arXiv cs.CL · arxiv.org