arXiv cs.AI· Georgios Alexopoulos, Nikolaos Alexopoulos, Diomidis Spinellis, Dimitris Mitropoulos·· 13 小时前AI 评分58
AI 过于顺从：通过偏见注入操纵自动化代码审查
When AI Agrees Too Much: Manipulating Automated Code Review via Bias Injection
AI 导读
研究考察 LLM 自动化代码审查（ACR）是否会被上下文偏见注入操纵，从而成为供应链攻击路径。作者先在 6 个 LLM、五种表述框架条件下确认框架效应普遍存在，再在 20 个真实项目的 33 个 CVE 上测试 Claude Code 和 CodeRabbit 两条流水线：模板式攻击无效甚至适得其反，而 LLM 辅助的迭代精炼攻击在 32/33（97%）案例中成功。
正文 · 原文
Abstract:Automated Code Review (ACR) systems integrating Large Language Models (LLMs) are increasingly adopted in software development workflows, ranging from interactive assistants to autonomous agents in CI/CD pipelines. In this paper, we study how LLM-based vulnerability detection in ACR is affected by the framing effect: the tendency to let the presentation of information override its semantic content in forming judgments. We examine whether adversaries can exploit this through contextual-bias injection (crafting PR metadata to bias ACR security judgments) as a supply-chain attack vector against real-world ACR pipelines. To this end, we first conduct a large-scale exploratory study across 6 LLMs under five framing conditions, establishing the framing effect as a systematic and widespread phenomenon in LLM-based vulnerability detection.
We then design a realistic and controlled experimental environment, evaluating 33 CVEs across 20 real-world projects and two popular ACR pipelines (Claude Code and CodeRabbit), to assess the susceptibility of real-world ACR pipelines to vulnerability re-introduction attacks. We employ two attack strategies: a template-based attack inspired by prior related work, and a novel LLM-assisted refinement attack. We find that template-based attacks are ineffective and may even backfire, as direct biasing attempts raise suspicions. Our refinement attack, on the other hand, is successful in 32/33 (97%) cases, exploiting a fundamental asymmetry: attackers can iteratively refine attacks against a local clone of the review pipeline, while defenders have only one chance to detect them. Overall, our findings highlight the dangers of over-relying on ACR and stress the importance of human oversight and contributor trust in the development process.
| Comments: | To appear at the 49th International Conference on Software Engineering (ICSE 2027) |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2603.18740 [cs.SE] |
| (or arXiv:2603.18740v5 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2603.18740 arXiv-issued DOI via DataCite |
Submission history
From: Georgios Alexopoulos [view email]
[v1]
Thu, 19 Mar 2026 10:40:27 UTC (350 KB)
[v2]
Thu, 23 Apr 2026 15:04:19 UTC (466 KB)
[v3]
Sat, 12 Sep 2026 16:58:07 UTC (513 KB)
[v4]
Wed, 23 Sep 2026 12:12:39 UTC (513 KB)
[v5]
Tue, 27 Oct 2026 14:45:00 UTC (508 KB)
来源：arXiv cs.AI · arxiv.org