arXiv cs.CL· Ming Zhong, Stacy Li, Nicholas Carlini, Matthew Jagielski·· 13 小时前AI 评分58
Agent4Sci：前沿智能体能否在真实科学 ML 任务上提升已发表最优模型
Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
AI 导读
研究者提出 Agent4Sci，一个覆盖 11 个领域、20 个科学 ML 基准的长程测试平台，用于衡量前沿编码智能体能在真实条件下把已发表最优模型提升多少。Claude Opus 取得 10.4% 的平均相对提升且无运行被标记为学术不端，但轨迹分析显示这些收益几乎全部来自 ML 工程技巧而非研究级洞察，并暴露出新颖性误判、资源利用不足和迅速陷入局部工程搜索三类局限。
正文 · 原文
Abstract:Frontier coding agents are increasingly framed as end-to-end automated researchers, and ML for science offers a demanding test of that promise: improving a scientific model often requires domain insight beyond routine ML engineering. We introduce Agent4Sci, a long-horizon testbed for measuring how far today's agents can improve the published best models on scientific ML benchmarks, spanning 20 benchmarks across 11 domains. Rather than ranking agents on a leaderboard, Agent4Sci asks what kinds of progress agents can make on these tasks under in-the-wild conditions, and where they fall short. Empirically, today's frontier agents advance the published best on every benchmark, with Claude Opus reaching 10.4% mean relative improvement and no run flagged for academic misconduct. These gains, however, come almost entirely from a bag of ML-engineering tricks rather than research-level insight. Trajectory analysis surfaces three recurring limitations in this setting: novelty miscalibration, resource under-utilization, and rapid collapse into local engineering sweeps. Lightweight plug-in skills for research ideation, experiment management, and scheduled checklists raise Claude Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%, while yielding more diverse, domain-relevant interventions. Agent4Sci gives the community a way to track and push frontier agents from ML engineering toward domain insight on real-world ML-for-science problems. Code and data are available on GitHub (this https URL) and Hugging Face (this https URL).
| Comments: | NeurIPS 2026 E&D Track |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.28763 [cs.AI] |
| (or arXiv:2610.28763v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28763 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ming Zhong [view email]
[v1]
Tue, 20 Oct 2026 18:03:48 UTC (11,505 KB)
来源：arXiv cs.CL · arxiv.org