AI ROI：为什么问题不在智能体本身
AI ROI: Why the Agent Isn't the Answer
Tessl 两场演讲指出，多数 AI 智能体在生产环境表现不佳，并非模型有误，而是其可见数据不完整、结果缺乏度量。Rob Willoughby 认为持久回报来自智能体周边的环境——上下文、评估与验证，而非更快的工具本身；Streambased CEO Tom Scott 则称智能体受限于数据可见性，缺失的集成与延迟更新形成盲点，降低准确率与自动化率，且更清晰的业务视图往往比更大的上下文窗口更有效。
Most AI agents underperform in production not because the model is wrong, but because the data around it is incomplete and the results are unmeasured. Two talks on the two constraints that actually limit agent ROI: what your agent can see, and whether you can tell if it's working.
Agenda
- 18:00 Venue opens
- 18:30 Talk 1: The ROI Isn't in the Agent by Rob Willoughby
- 19:00 Talk 2: Data Blindspots: The Hidden Killer of AI ROI by Tom Scott, CEO, Streambased
- 19:30 Networking
- 20:30 THE END
The ROI Isn't in the Agent
Many agent ROI numbers are fiction. They come out of the spreadsheet every engineering leader already knows: hours saved per developer, multiplied by headcount, multiplied by a hopeful factor. The trouble is that this treats the agent as the thing you're buying, and treats the saving as something you bank once and move on from.
This talk argues that the return that actually lasts sits one layer up, in the environment around the agent: the context it works from, the evaluations, the verification that decides whether its output is any good. Savings from a faster tool get spent and don't recur. Money put into the harness keeps paying back, because each piece you build makes the next one cheaper and the data you throw off along the way accumulates.
It also looks at why none of that is bankable until you can measure it, and why getting from vibes to metrics is where an honest ROI story has to start.
Rob Willoughby, Member of Technical Staff, AI Research at Tessl
Rob works on evaluation research at Tessl, designing and running large-scale assessments of how coding agents behave in real-world codebases. His work focuses on figuring out what "good" looks like when an AI agent works with your code from eval design and rubric systems to understanding where models (and infrastructure) break down at scale.
Data Blindspots: The Hidden Killer of AI ROI
Why do so many AI agent projects fail to deliver meaningful ROI?
The usual answers focus on models, prompts, and tooling. But in practice, the biggest limitation is often much simpler: agents can't see enough of the business.
Most organisations give agents access to a handful of data sources and expect them to make decisions using a fragmented, incomplete view of reality. Every missing integration, delayed update, or inaccessible dataset becomes a blindspot. And every blindspot reduces accuracy, confidence, automation rates, and ultimately ROI.
In this talk, I'll explore why agent performance is fundamentally constrained by data visibility. We'll look at how blindspots emerge, why traditional integration approaches struggle to keep up with the growing demand for agent-ready data, and how modern event-driven architectures can make business context immediately available to AI systems.
We'll also examine the relationship between visibility and efficiency: why more data isn't always better, how context minimisation improves agent performance, and why the most successful agent deployments are often the ones that provide the clearest view of reality rather than the largest context windows.
Low ROI agents aren't stupid. They're blind.
Tom Scott, CEO, Streambased
Tom Scott runs Streambased, the company unifying Apache Kafka and Apache Iceberg for AI projects. Tom has more than 20 years of experience in distributed systems and data infrastructure working as a customer and vendor for some of the most exciting problems in event streaming and big data.
This event is brought to you as part of the AI Native Dev Community. Consider subscribing to the Mailing List, Podcast, and joining our Discord Community
来源：Tessl Blog · tessl.io