arXiv cs.AI· Gil Pasternak, AJ Langley, Tom Yotam, Ellie Kitanidis·· 4 小时前AI 评分38
编码智能体能否编写并转译单元测试
Can Coding Agents Write and Transpile Unit Tests
AI 导读
研究基准测试了三款主流编码智能体(均搭配前沿模型并启用高推理强度),发现其在传统覆盖率指标上远超人类编写的测试套件,但优势并不均衡:测试质量随项目规模增大而明显下降,且优势很大程度来自写出的测试数量更多。智能体跨编程语言转译测试在整体指标上看似不错,但结果仍不稳定、不可靠。
正文 · 原文
Abstract:Coding assistants built on Large Language Models (LLMs) have become a staple of modern software development, increasingly trusted to execute workflows and modernize code end-to-end. Much of this trust rests on passing tests, which are often written by the coding agents themselves. In this work, we examine the ability of these agents to both write reliable tests and translate them between programming languages. We benchmark three popular coding agents, each evaluated with a native frontier model and high reasoning effort. Verifying across a broad suite of metrics, we find that coding agents far exceed human written suites on traditional measures of coverage. This advantage, however, is uneven: agent test quality degrades sharply with project size, and much of the advantage lies in a greater number of written tests. We also show that agentic test translation---while seemingly strong in aggregate---remains inconsistent and unreliable.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.21797 [cs.SE] |
| (or arXiv:2611.21797v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21797 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Gil Pasternak [view email]
[v1]
Mon, 16 Nov 2026 19:16:19 UTC (645 KB)
来源:arXiv cs.AI · arxiv.org