arXiv cs.AI· Yuwei Wang, Xingwang Liang, Yu Xie, Haotian Chi, Haijun Geng·· 12 小时前AI 评分34
RiskCueBench：大语言模型安全行为在意图可恢复后仍会失准
Recoverability-Controlled Safety Miscalibration in Large Language Models
AI 导读
研究者提出 RiskCueBench，一个可恢复性控制基准，用 160 个潜在意图和 640 条抽象非操作性表面提示词，把意图理解与响应层安全行为分开评估。在两种表面形式均可恢复的配对条件下，上下文具体化程度提高会使拒绝率下降 0.175、全局安全得分下降 0.440，不安全配合上升 0.107。结果表明即使意图已被恢复，模型安全行为仍会随表面形式失准。
正文 · 原文
Abstract:Safety-aligned language models are expected to refuse harmful requests while remaining useful for benign ones. However, because many evaluations score only the final response, they often conflate intent understanding with response-level safety behavior. To address this, we introduce RiskCueBench, a recoverability-controlled benchmark that separates these two stages by first measuring whether a model can successfully recover a high-level intent and then evaluating its safety behavior on the same prompt. RiskCueBench contains 160 latent intents and 640 abstract non-operational surface prompts across eight risk categories. The benchmark covers 25 completed model groups and 32,000 successful model outputs with final rubric-adjudicated labels. When conditioning on pairs where both surface forms are recoverable, we find that increased contextualized specificity induces a shift toward less safe behavior. Specifically, paired bootstrap estimates show that the refusal rate and global safety score decrease by 0.175 and 0.440, respectively. Furthermore, mixed effects modeling corroborates a 0.107 increase in unsafe cooperation. These results show that the safety behavior can be miscalibrated to the surface form even after the intent has been recovered, motivating safety evaluations that report response behavior conditional on semantic recoverability.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22425 [cs.CR] |
| (or arXiv:2610.22425v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22425 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuwei Wang [view email]
[v1]
Thu, 15 Oct 2026 05:02:39 UTC (908 KB)
来源：arXiv cs.AI · arxiv.org