arXiv cs.AI· Nan Huang, Mario Tapia-Pacheco, Kun Zhou, Yiming Huang, Kevin Jos\'e Barrientos D\'iaz, Tiffany Amariuta, Jingbo Shang·· 14 小时前AI 评分38
DISCERN 基准：AI 智能体能像科学家一样工作并引导发现吗？
DISCERN: Can AI Agents Work Like Scientists and Guide Discovery?
AI 导读
研究团队推出 DISCERN 基准，基于真实公开数据集评估 AI 智能体自动化科研流程的三个层级：数据完整性、分析验证与假设生成。在 203 项任务、8 个生命科学赛道和 8 个模型上，智能体仅在 60.8% 的 Level 1、34.2% 的 Level 2 和 0.6% 的 Level 3 评估中拿到满分。结果表明当前智能体尚不具备可靠的自主分析与发现能力，但可定位监督式科研辅助的机会。
正文 · 原文
Abstract:Reliable automated research requires agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially reducing routine scientific workload while allowing scientists to focus on interpretation and discovery. Existing benchmarks often only assess analytical task completion or hypothesis generation separately rather than testing whether reliable evidence supports valid and novel claims. We introduce DISCERN (Data Integrity and Scientific Capability: Evidence, Reasoning, and Novelty), a controlled benchmark on real, publicly available datasets that evaluates three key levels of an automated research workflow. The first two levels test data integrity and analysis verification under confounds and tool traps, while the third tests hypothesis generation and revision under adversarial review, including counterfactual cases in which evidence consistent with real data and documented scientific phenomena conflicts with established expectations, motivating alternative explanations and testable hypotheses. Across 203 tasks, eight life-science tracks, and eight models, DISCERN shows that strong aggregate performance can mask level-specific weaknesses. Agents earn perfect scores in only 60.8% of Level 1, 34.2% of Level 2, and 0.6% of Level 3 evaluations, with penalties attributed to rejection of sound data, failure to carry recognized limitations into conclusions, and wide variation in hypothesis production. Cross-track rankings by token and code use are substantially more stable than rankings by evidence judgment, suggesting greater consistency in computational effort than in evidence-based reasoning. These profiles identify opportunities for supervised scientific assistance, but current agents do not yet demonstrate reliable autonomous analysis or discovery. Code and data: this https URL
| Comments: | Project website: this https URL. Updated to include the project website and planned life-science verification evaluation |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.33357 [cs.AI] |
| (or arXiv:2609.33357v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.33357 arXiv-issued DOI via DataCite |
Submission history
From: Mario Tapia-Pacheco [view email]
[v1]
Sun, 27 Sep 2026 08:29:32 UTC (1,564 KB)
[v2]
Mon, 26 Oct 2026 05:50:43 UTC (1,564 KB)
来源：arXiv cs.AI · arxiv.org