arXiv cs.MA· Ke Chen, Xucheng Yu, Yufei Zhou, Haohan Wang·· 4 天前AI 评分31
提示词稳定性为何重要：通用系统中自动生成提示词的评估与优化
Prompt Stability Matters: Evaluating and Optimizing Auto-Generated Prompt in General-Purpose Systems
AI 导读
研究提出以"语义稳定性"衡量提示词在重复执行下模型响应的一致性，并据此构建了首个稳定性感知的通用提示词生成系统，利用稳定性反馈迭代提升提示词质量与系统性能。作者还通过分析系统内部结构依赖，论证提示词稳定性是系统级有效执行的必要条件。在通用与领域特定任务上，该框架同时提升了准确率和输出一致性，UI 与源码已公开。
正文 · 原文
Abstract:Automatic prompt generation plays a crucial role in enabling general-purpose multi-agent systems to perform diverse tasks autonomously. Existing methods typically evaluate prompts based on their immediate task performance, overlooking the intrinsic qualities that determine their reliability. This outcome-centric view not only limits interpretability but also fails to account for the inherent stochasticity of large language models (LLMs). In this work, we bring attention to prompt stability-the consistency of model responses across repeated executions-as a key factor for building robust and effective prompt generation systems. To quantify this, we propose semantic stability as a criterion for assessing the response consistency of prompts. Based on the proposed metric, we developed the first stability-aware general-purpose prompt generation system that leverages stability feedback to iteratively enhance both prompt quality and system-level performance. Furthermore, we establish a logical chain between prompt stability and task success by analyzing the structural dependencies within our system, proving stability as a necessary condition for effective system-level execution. Empirical results across general and domain-specific tasks demonstrate that our stability-aware framework improves both accuracy and output consistency. By shifting the focus from one-off results to persistent reliability, our work offers a new perspective on prompt design and contributes practical tools for building more trustworthy general-purpose systems. The UI and source code for our system are publicly available at: this https URL.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2505.13546 [cs.AI] |
| (or arXiv:2505.13546v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2505.13546 arXiv-issued DOI via DataCite |
Submission history
From: Ke Chen [view email]
[v1]
Mon, 19 May 2025 03:28:33 UTC (527 KB)
[v2]
Tue, 13 Oct 2026 18:09:51 UTC (626 KB)
来源：arXiv cs.MA · arxiv.org