PersonaCodeBench：评估编码智能体在人格驱动人机交互会话中的表现
PersonaCodeBench: Evaluating Coding Agents in Persona-driven Human-Interactive Sessions
研究团队提出 PersonaCodeBench，将用户角色、技术能力、交互偏好与实际约束转化为可验证的个性化需求，用于评估编码智能体在多轮交互中的表现。该基准通过控制器管理用户模拟器响应与需求披露，并由完成验证器和个性化验证器分别评估代码正确性与人格需求满足度，联合通过需同时满足预算与有效性条件。研究对主要前沿智能体进行了评测并建立了排行榜。
Authors:Haixin Wang, Jianheng Hou, Hongyu Shen, Wenzhe Fan, Wei-Chieh Huang, Qianyu Zhu, Weichen Li, Taoran Li, Yuhang Yao, Ruohan Wang, Zhejian Peng, Wanting Jiang, Yifan Chen, Shushu Liang, Haoming Li, Defu Cao, Xu Hu, Siyang Wu, Chen Yang, Haodi Hu, Haichao Zhang, Cong Wang, Zhiyuan Yu, Weizhi Zhang, Yifan Wang, Jintao Huang, Yizhou Sun, Yijun Wang, Yuexing Hao, Xiaomin Li
Abstract:Coding agents must deliver correct software while respecting users' expertise, preferences, and working conditions. Existing benchmarks increasingly incorporate interaction, yet personalized requirement fulfillment remains underexplored. We introduce PersonaCodeBench, which turns user characteristics into verifiable obligations in persona-driven multi-turn coding sessions. Our persona pool spans roles, technical abilities, interaction preferences, and practical constraints, yielding task-grounded evolved requirements for implementation, permissions, testing, explanation, and delivery. At runtime, a controller governs user simulator responses and requirement disclosure, using online requirement evaluation to provide feedback and advance the interaction. Requirements emerge progressively, with their content, disclosure conditions, and acceptance criteria fixed before evaluation. A completion verifier and a personalization verifier separately assess native correctness and persona fulfillment. Joint pass requires both adherence to the declared budget and compliance with validity conditions. We evaluate all major frontier agents, establish a comprehensive leaderboard, and conduct detailed experimental analyses to validate effectiveness. PersonaCodeBench supports reproducible evaluation of coding agents' ability to meet explicit, personalized user expectations alongside coding objectives.
| Comments: | 40 pages, 9 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.05739 [cs.AI] |
| (or arXiv:2611.05739v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05739 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haixin Wang [view email]
[v1]
Tue, 3 Nov 2026 23:27:20 UTC (3,500 KB)
来源：arXiv cs.AI · arxiv.org