arXiv cs.AI· Jenny T. Liang, Mihika Bairathi, Wayne Chi, Ameet Talwalkar, Nishant Subramani, Valerie Chen·· 14 小时前AI 评分38
开发者如何修改 AI 生成的代码？LLM 能否推断其编辑意图：来自 53.6K 次开发者编辑的洞察
How Do Developers Edit AI-Generated Code, and How Well Do LLMs Infer Their Edits? Insights from 53.6K Developer Edits
AI 导读
研究团队构建 DECODE 数据集，收录来自 1K+ 开发者的 53.6K 条 IDE 内真实代码编辑记录，用于分析开发者何时、为何、如何修改 AI 生成的代码。结果显示开发者对 AI 生成代码的编辑投入很少、放弃很快；预测代码编辑对前沿模型和微调模型都很困难，前沿模型最能反映开发者意图但易过度编辑并回退开发者手写改动，微调可提升相似度却仍未追上前沿模型。
正文 · 原文
Abstract:Imperfections in AI-generated code require that software developers modify generations through manual code edits. However, little is known about how developers edit AI-generated code, and to what extent LLMs can infer implicit developer intent from this data, given the lack of high-quality, realistic code editing data. This information is essential for designing and building usable AI-developer interactions. To address this gap, we introduce DECODE (Developer Edits of Code Dataset), a dataset of 53.6K real-world in-IDE code edits of AI-generated code sourced from 1K+ developers. First, we use DECODE to analyze when, why, and how AI-generated code is edited. Developers invest little effort in editing code and abandon AI-generated code quickly, thereby motivating the need for LLMs to understand developer intents through their edits. Therefore, we next use DECODE to understand to what extent LLMs could infer developer edits. Predicting code edits is a challenging task for both frontier and fine-tuned models. Frontier models generate edits that best reflect developer intents, but are prone to over-editing and reverting developer-written edits. Fine-tuning LLMs can improve their ability to predict edits that are more similar to developers' as well as better reflect developer intent, but do not fully bridge the gap to frontier models. Together, our results demonstrate the unique insights on developer and LLM behavior that can be obtained through studying realistic code edit data to promote more better developer-AI interactions.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.25130 [cs.SE] |
| (or arXiv:2607.25130v3 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2607.25130 arXiv-issued DOI via DataCite |
Submission history
From: Jenny Liang [view email]
[v1]
Mon, 27 Jul 2026 22:50:05 UTC (4,732 KB)
[v2]
Mon, 14 Sep 2026 16:31:50 UTC (4,732 KB)
[v3]
Tue, 27 Oct 2026 21:18:47 UTC (5,511 KB)
来源：arXiv cs.AI · arxiv.org