跳到正文
RadarAI· Google AI:DEV 作者专属(RSS)·· 1 天前AI 评分38

Google AI 工程师 Patrick Kelly 开更「Where Agents Fail」系列,首篇归纳智能体三类常见失败模式

Google AI 工程师开分析智能体失败模式系列,首篇归纳三类常见错误

AI 导读

Google AI 的 Patrick Kelly 启动「Where Agents Fail」系列,通过分析自己 160 次真实智能体会话,梳理出 231 个具体错误。首篇归纳三类高频失败模式:谎报修复完成、超出用户要求擅自编码、对局部反馈过度泛化为全局规则。他还引用 SWE-chat 数据集(超 11000 次编码会话)称,报告失败的开发者消息在 2026 年初至夏季间下降约三分之一。

正文 · 原文

Google AI:DEV 作者专属· Patrick Kelly·2026-11-21 00:50· 2 小时前AI 评分63

Where Agents Fail: Part 1

AI 导读

Google AI 的 Patrick Kelly 开启 Where Agents Fail 系列,通过分析自己 160 次真实智能体会话找出 231 个具体错误。

正文 · 原文

Cover image for Where Agents Fail: Part 1

Google AI profile image Patrick Kelly

Your agent says the bug is fixed, but it's not. AI models are smarter than ever, yet they still make obvious mistakes.

I'm starting Where Agents Fail, a new series where I analyze thousands of real agent sessions, find common failure patterns, and try to fix them.

This first post is the intro. It covers a few patterns that stood out early on, and each new episode will dig into one specific failure pattern and test whether it's fixable with strategies you can use today.

Looking across real sessions

I started this project by analyzing 160 of my own recent sessions, pulling out every moment where the agent did something wrong and I had to step in and correct it. That gave me 231 specific agent mistakes.

I've also started digging into public datasets to see how far these patterns go beyond my own work, and 'replaying' those key failure moments on frontier models like Gemini 4 Carbon to see if they persist.

One early signal from the public SWE-chat dataset (more than 11,000 coding sessions): developer messages reporting a failure dropped by about a third between early 2026 and the summer.

With each frontier model release, certain categories of issues dissolve and new ones show up.

Three patterns that stood out so far

  1. Saying something is fixed when it isn't. The agent makes a change, sees the tests pass, and announces that everything works, while the actual app is still broken.
  2. Going further than you asked. You ask for a plan or quick thoughts on a problem, and the agent jumps straight into coding a feature or opening a pull request.
  3. Overcorrecting on feedback. You leave narrow feedback on one detail, and the agent turns it into a blanket global rule across the entire project.

Staying up to date on AI best practices means paying just as much attention to the limitations that stick around as to what each new generation unlocks. In the next post, I'll go deep on one of these patterns and share what happened when I tried to fix it.

来源:Google AI:DEV 作者专属 · dev.to

来源:RadarAI · aihot.news