arXiv cs.AI· Junlong Liu, Haobo Wang, Weiqi Luo, Xiaojun Jia·· 11 小时前AI 评分38
绕过提示词防线：SMT 利用模拟审核轨迹越狱函数调用 LLM
Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation Traces
AI 导读
研究者提出黑盒攻击框架 SMT，通过构造模拟合法审核流程的多轮轨迹，把安全拒答当作执行失败来迭代优化，从而越狱函数调用类 LLM。在五家厂商的商业 LLM 和两个标准化安全基准上，SMT 以接近最少的查询次数取得最高平均攻击成功率和 HarmScore，显著优于现有基线。该工作指出仅靠提示词层面的清洗无法防御工具调用系统，需要在 schema、参数、工具输出和对话状态上做上下文感知校验。
正文 · 原文
Abstract:Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs). While prior work has primarily studied attacks and defenses at the prompt level, we show that this prompt-centric paradigm overlooks a structural vulnerability in stateful, function-calling environments. In such applications, developer-defined schemas, structured arguments, and untrusted tool outputs are interleaved into a single shared model context. This architecture expands the attack surface by blurring the boundary between trusted control logic and untrusted data, allowing adversarial intent to be distributed across a multi-turn execution path. We exploit this architectural flaw through SMT, a black-box attack framework based on Simulated Moderation Traces. Departing from purely prompt-based interactions, SMT constructs a multi-turn trajectory that simulates a legitimate moderation-auditing workflow. Within this trajectory, a fabricated moderation frame leverages red-team testing as a pretext to elicit harmful generations. The subsequent validation feedback treats safety refusals as execution failures, prompting refinements that gradually weaken the model's safety constraints and ultimately trigger harmful outputs. Extensive empirical evaluations on prominent commercial LLMs from five different providers across two standardized safety benchmarks show that SMT consistently achieves the highest average attack success rate and HarmScore while requiring a near-minimal number of queries, substantially outperforming existing baselines. These findings demonstrate that prompt-level sanitization alone is fundamentally insufficient for defending tool-enabled LLM systems and highlight the urgent need for context-aware validation across schemas, arguments, tool outputs, and accumulated conversation state. The code is available at this https URL.
| Comments: | Accepted to the 2027 IEEE Symposium on Security and Privacy (S&P) |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.00481 [cs.CR] |
| (or arXiv:2607.00481v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2607.00481 arXiv-issued DOI via DataCite |
Submission history
From: Haobo Wang [view email]
[v1]
Wed, 1 Jul 2026 06:08:07 UTC (1,150 KB)
[v2]
Wed, 4 Nov 2026 05:40:42 UTC (1,639 KB)
来源：arXiv cs.AI · arxiv.org