arXiv cs.AI· Tianjian Li, Chuanyang Jin, Seungone Kim, Tong Chen, John Hoffman, Yoram Bachrach amd Pedro Rodriguez, Ping Yu, Eryk Helenowski, Swarnadeep Saha, Jack Lanchantin, Weizhe Yuan, Ilia Kulikov, Jason Weston·· 14 小时前AI 评分42
让 Autoresearch 对齐：智能体与人类的共同改进
Aligning Autoresearch: Co-Improvement between Agents and Humans
AI 导读
研究团队从四方面推进 Autoresearch 智能体与人类的共同改进：实证发现人类在基准创建尤其是构思阶段仍起关键作用；提出 HackBench 基准，显示当前智能体对齐不足、难以充分定位 reward hacking；并给出两种训练方法——用人类模拟器作为 RL 代理来提升人机交互，以及用与人类专家对齐的 rubric 做 RL 改进模型文本生成与智能体沟通。
正文 · 原文
Authors:Tianjian Li, Chuanyang Jin, Seungone Kim, Tong Chen, John Hoffman, Yoram Bachrach amd Pedro Rodriguez, Ping Yu, Eryk Helenowski, Swarnadeep Saha, Jack Lanchantin, Weizhe Yuan, Ilia Kulikov, Jason Weston
Abstract:Autoresearch agents can now attempt long-horizon AI research end-to-end, i.e., self-improvement. However, best results are still obtained by research agents interacting with human researchers, and improving AI's ability to perform this co-research will only reinforce this effect---a process termed co-improvement (Weston and Foerster, 2025). Further, aligning with human expectations is likely a safer path for research agents, especially given recent agent reward hacking phenomena (OpenAI, 2026b). In this work we address these issues in four ways: (i) we empirically study the role of humans in directing autoresearch agents, focusing on the domain of benchmark creation, and find humans still play a pivotal role particularly in the ideation phase; (ii) we also develop a benchmark, HackBench, that shows current autoresearch agents are not aligned and are unable to achieve hack localization sufficiently; (iii) we present a method to train agents for improved human interaction by using human simulators as a proxy during reinforcement learning (RL); and finally (iv) we develop a method to improve model text generation, and hence agent communication, by RL with rubrics aligned with human experts. Together, our training and evaluation recipes provide a path toward more capable and aligned (co-)research agents.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.38882 [cs.AI] |
| (or arXiv:2610.38882v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38882 arXiv-issued DOI via DataCite |
Submission history
From: Jason Weston [view email]
[v1]
Wed, 28 Oct 2026 20:47:10 UTC (3,182 KB)
来源：arXiv cs.AI · arxiv.org