arXiv cs.CL· Fardin Ahsan Sakib, Ziwei Zhu, Ozlem Uzuner·· 12 小时前AI 评分38
模型自己知道：防御医疗多智能体系统的提示词注入攻击
The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection
AI 导读
针对医疗多智能体系统的提示词注入攻击在两种系统、两个模型家族和三个基准上单段注入成功率最高达 61%，且智能体协作并不能稳定防御。研究提出一种免训练防御：按临床实践将患者发现与诊断、建议、指令等临床判断分离，仅依据患者发现逐条核验判断，使决策只建立在有支持的判断上。
正文 · 原文
Abstract:Medical multi-agent systems, in which LLM agents discuss a case, are increasingly studied for clinical decision support. Prompt injection, text that an attacker adds to a model's input, can manipulate LLMs into wrong decisions. How it affects these systems, and how to protect them, remains unclear. Across two systems, two model families and three benchmarks, a single injected passage succeeds in up to 61% of cases, and collaboration does not consistently protect against it. Existing defenses help only partly: they miss injections that look like ordinary clinical text. Yet when asked to check the injected text against the patient's findings, a model that followed the injection usually rejects it. We therefore propose a training-free defense that verifies the clinical judgments behind the system's decision. Following clinical practice, it separates the patient's findings from clinical judgments, such as diagnoses, recommendations and instructions, and checks each judgment against the findings alone, so the decision rests only on supported judgments. Our defense outperforms all baselines on every benchmark, lowering attack success on MedQA to 4-7% while maintaining accuracy on cases without an attack, and remains effective with a medical model and against defense-aware and adaptive attacks.
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.22594 [cs.CL] |
| (or arXiv:2610.22594v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22594 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Fardin Ahsan Sakib [view email]
[v1]
Thu, 15 Oct 2026 19:43:57 UTC (260 KB)
来源：arXiv cs.CL · arxiv.org