arXiv cs.AI· Lindsey Ferris, Sierra Bonilla·· 10 天前AI 评分40
如何保障 AI 编写软件的质量：一个医疗平台案例研究
A Case Study in Assuring AI-Written Software
AI 导读
一项案例研究记录了一个由编码智能体构建、由无正式软件工程训练的操作者治理的生产级医疗平台。其工作流逐步演变为人类主导的元智能体系统：一个智能体写代码，其他智能体监督审查，项目规则沉淀经验。操作者发现测试、监控与审查智能体均可能出错，包括监控代理指标而非结果、审计静默失败、缺失检查从结果中消失，以及一次自动修复引发运营中断。
正文 · 原文
Abstract:Software-engineering agents can enable people without formal software training to build systems they could not otherwise implement and simultaneously can produce more code than even experts can meaningfully inspect. In both cases, exhaustive code review is not reliable as the sole basis for human control. We report a case study of a production healthcare platform built through coding agents and governed by an operator without formal software-engineering training. Over time, its workflow grew into a human-led meta-agent system where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The operator found that tests, monitors and reviewing agents used to supervise the system were fallible. Some monitors measured proxies rather than outcomes, some audits failed silently, missing checks disappeared from reported results and one automated repair caused operational disruption. In this case, human control depended on keeping the intended outcome, the evidence used to judge it, the agents' permissions and the final human decision were all tied to the same underlying objective.
| Comments: | Accepted to the NeurIPS 2026 Meta-Agents Workshop |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.08651 [cs.SE] |
| (or arXiv:2610.08651v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08651 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sierra Bonilla [view email]
[v1]
Tue, 6 Oct 2026 16:37:08 UTC (190 KB)
来源：arXiv cs.AI · arxiv.org