PersonaScienceBench：用多样化模拟研究者评估科学智能体
PersonaScienceBench: Evaluating Scientific Agents with Diverse Simulated Researchers
PersonaScienceBench 基准发布，包含来自五个基准、覆盖四个科学领域的 200 个任务，模拟研究者按 persona 画像提供上下文与反馈，任务目标与评分保持不变。在十八种模型配置上，任务表现、事实召回与自我重复呈现不同优势；GPT 6 Astra 的分析显示，即使最终得分相同，智能体在不同研究者画像下的澄清提问与工作流调整行为也可能不同。
Authors:Siyang Wu, Weizhi Zhang, Chen Yang, Esther Xu Fei, Jorin Overwiening, Xu Pan, Jing Tang, Yixuan He, Kelvin C. Mo, Peiwen Li, Defu Cao, Xu Hu, Haixin Wang, Taoran Li, Haoming Li, Zheyuan Deng, Jingwen Zhang, Pengsong Zhang, Haichao Zhang, Guangze Luo, Yi Daniel Lu, Zihuai He, Chenxi Qiu, Dianzhuo Wang, Zhejian Peng, Shushu Liang, Wenyi Lu, Yifan Wang, Jintao Huang, Yijun Wang, Xiaomin Li, Yuexing Hao
Abstract:Scientific agents collaborate with researchers whose expertise and preferences differ, but benchmarks usually hold the collaborator fixed. We introduce PersonaScienceBench: 200 tasks from five benchmarks across four scientific domains. Simulated researchers provide context and feedback according to persona profiles, while task objectives and grading remain fixed. Agents encounter researcher preferences only through conversation. Across eighteen modelconfigurations, task performance, fact recovery, and self-repetition reveal different strengths. Detailed GPT 6 Astra analyses show distinct agent responses across researcher profiles; interaction traces illustrate targeted clar ification and workflow revision. These behaviors can differ even when final scores match. PersonaScienceBench therefore evaluates how agents collaborate with diverse researchers alongside whether they complete the task.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32106 [cs.AI] |
| (or arXiv:2610.32106v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32106 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Siyang Wu [view email]
[v1]
Fri, 23 Oct 2026 03:11:01 UTC (237 KB)
来源：arXiv cs.AI · arxiv.org