跳到正文
arXiv cs.CL· Hussein Jawad, Nicolas J-B Brunel·· 9 小时前AI 评分35

PSM:用 LLM 引导的黑盒优化最小化系统提示词敏感性

PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimization

AI 导读

研究者提出 PSM 框架,通过"盾牌附加"为系统提示词增加一层保护性文本,并将提示词加固形式化为效用约束优化问题,由 LLM 作为优化器搜索 SHIELD 空间。

正文 · 原文

View PDF HTML (experimental)

Abstract:System prompts are critical for guiding the behavior of Large Language Models (LLMs), yet they often contain proprietary logic or sensitive information, making them a prime target for extraction attacks. Adversarial queries can successfully elicit these hidden instructions, posing significant security and privacy risks. Existing defense mechanisms frequently rely on heuristics, incur substantial computational overhead, or are inapplicable to models accessed via black-box APIs. This paper introduces a novel framework for hardening system prompts through shield appending, a lightweight approach that adds a protective textual layer to the original prompt. Our core contribution is the formalization of prompt hardening as a utility-constrained optimization problem. We leverage an LLM-as-optimizer to search the space of possible SHIELDs, seeking to minimize a leakage metric derived from a suite of adversarial attacks, while simultaneously preserving task utility above a specified threshold, measured by semantic fidelity to baseline outputs. This black-box, optimization-driven methodology is lightweight and practical, requiring only API access to the target and optimizer LLMs. We demonstrate empirically that our optimized SHIELDs significantly reduce prompt leakage against a comprehensive set of extraction attacks, outperforming established baseline defenses without compromising the model's intended functionality. Our work presents a paradigm for developing robust, utility-aware defenses in the escalating landscape of LLM security. The code is made public on the following link: this https URL
Comments: accepted at the Trustworthy Agentic AI Workshop @ AAAI 2026 (Singapore, Jan 27, 2026). Workshop PDF: this https URL
Subjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL)
Cite as: arXiv:2511.16209 [cs.CR]
  (or arXiv:2511.16209v3 [cs.CR] for this version)
  https://doi.org/10.48550/arXiv.2511.16209

arXiv-issued DOI via DataCite

Submission history

From: Hussein Jawad [view email]
[v1] Thu, 20 Nov 2025 10:25:45 UTC (36 KB)
[v2] Sun, 1 Feb 2026 16:18:00 UTC (36 KB)
[v3] Tue, 17 Nov 2026 13:51:44 UTC (2,783 KB)

来源:arXiv cs.CL · arxiv.org