卡内基梅隆与微软发布 CRBugBench：最佳代码审查 Agent 检出 71.9% 已知 bug
卡内基梅隆与微软论文发布 CRBugBench：最佳代码审查 Agent 检出 71.9% 已知 bug
卡内基梅隆与微软论文提出 CRBugBench，包含 1,800 个代码审查任务，每个 PR 引入一个人工确认并修复的 bug。最佳模型 claude-opus-5 在前 5 条评论中检出 71.9% 的 bug，高于 5 个月前 opus-4.6 的 47.7%，但在某公司私有仓库上下降最多 25.5%。作者建议审查 Agent 与人类审查者配合使用，并为其提供仓库自有知识。
Rohan Paul· @rohanpaul_ai · X·2026-10-14 00:08· 55 分钟前AI 评分50
AI 导读
卡内基梅隆与微软论文提出 CRBugBench，包含 1,800 个代码审查任务，每个 PR 引入一个人工确认并修复的 bug。最佳模型 claude-opus-5 在前 5 条评论中检出 71.9% 的 bug，高于 5 个月前 opus-4.6 的 47.7%，但在某公司私有仓库上下降最多 25.5%。作者建议审查 Agent 与人类审查者配合使用，并为其提供仓库自有知识。
正文
New Carnegie Mellon and Microsoft paper finds that the best AI code reviewer caught 71.9% of known bugs in its top 5 comments, still too few to remove human reviewers.
Agents now write code faster than people can review it, tempting teams to let AI review it too. Older review benchmarks scored AI against human comments, which can be nitpicks or miss real bugs.
They built CRBugBench, 1,800 code review tasks where each pull request introduced a bug that humans confirmed and fixed. The agent reviews the original change in a sandbox, and AI judges check whether its top findings name that bug.
The best agent, claude-opus-5, caught 71.9% of bugs in its top 5 comments, up from 47.7% for opus-4.6 5 months earlier. Performance dropped up to 25.5% on 1 company's private repos, where bugs more often need project context.
Use review agents alongside human reviewers, and give them your repo's own knowledge, especially on large or context-heavy changes.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news