arXiv cs.AI· Aditya Sivakumar, Peter Ciaccia, Hassaan Qadir, Ian Hsu, Ashu Singhal·· 12 小时前AI 评分31
BenchBench-Cloning：评估大语言模型的分子克隆设计与实验方案开发能力
BenchBench-Cloning: Evaluating Molecular Cloning Design and Protocol Development
AI 导读
研究团队推出 BenchBench-Cloning 基准，包含 65 个分子克隆任务，分为 50 个实验方案开发任务和 15 个构建体设计任务，方案任务通过克隆模拟器执行评估。在八个系统上重复测试中，GPT-6 Astra 得分最高，为 30.6%，其余系统在 8.6% 至 22.2% 之间，且 best-of-ten 采样下基准仍未饱和。
正文
Abstract:Molecular cloning is one of the most routine wet-lab workflows, yet existing evaluations of language models in biology mostly test knowledge recall or isolated steps rather than end-to-end experimental planning. We introduce BenchBench-Cloning, a benchmark for large language models comprising 65 molecular cloning tasks grounded in sequence records, laboratory notebooks, and published experimental studies. Planning a molecular cloning experiment requires defining a set of constructs needed for a scientific objective and a construction route to produce them. BenchBench-Cloning contains 50 protocol-development tasks, in which agents produce an executable protocol for a specified construct, and 15 construct-design tasks, in which agents determine the constructs a scientific objective calls for. Protocol-development tasks are evaluated by execution in a cloning simulator; construct-design tasks are evaluated with task-specific deterministic checks and, where needed, rubric-based LLM-as-a-judge evaluation. Across eight systems with repeated attempts per task, GPT-6 Astra scores highest at 30.6%, with the others between 8.6% and 22.2%, and the benchmark remains unsaturated under best-of-ten sampling. As integrating individual cloning decisions into complete, valid plans remains challenging even for frontier models, we expect BenchBench-Cloning to serve as a useful benchmark for tracking progress in experimental planning.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33355 [cs.AI] |
| (or arXiv:2610.33355v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33355 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aditya Sivakumar [view email]
[v1]
Fri, 23 Oct 2026 22:51:29 UTC (208 KB)
来源：arXiv cs.AI · arxiv.org