arXiv cs.AI· Tongxi Wu, Yuyang Cheng, Jiacheng Xian, Menglin Yang, Kai Chen, Jian Zhang, Yang Gao·· 23 小时前AI 评分42
视觉语言模型的场景状态安全漏洞:SceneSafetyBench 与 S3TA 攻击
Reading Scene States: Uncovering Safety Failures in Vision-Language Models
AI 导读
研究识别出视觉语言模型的“场景状态安全漏洞”,并构建 SceneSafetyBench(375 个图文问答对,覆盖 Initial、Medium、Ultimate 三种场景状态与五种视觉呈现),八款 VLM 在场景状态输入下的攻击成功率为 45.6%–72.3%。
正文 · 原文
Abstract:Vision-language models (VLMs) can provide harmful assistance by interpreting visual events, even without an explicit request for the underlying harmful objective. We identify this \textbf{scene-state safety vulnerability} and introduce \textbf{SceneSafetyBench}, comprising 375 image--question pairs spanning Initial, Medium, and Ultimate scene states and five visual presentations. Across eight VLMs, scene-state inputs elicit harmful assistance with attack success rates (ASRs) of 45.6--72.3\%. Comparative analysis across three VLMs shows that scene-state inputs most closely resemble QR-related images in their layer-wise safety-related signal profiles, while exhibiting lower average late-layer readouts and distinct signal dynamics during generation. To turn this vulnerability into a systematic attack, we propose \textbf{S3TA}, which translates harmful objectives into scene states, queries image--question pairs independently, and assembles the responses. On seven MM-SafetyBench categories across eight open-weight and commercial VLMs, S3TA achieves a model-average ASR of \textbf{71.2\%, exceeding the strongest baseline by 6.1 percentage points}. Text-side filters and visual moderation provide limited protection. These findings highlight the need to safeguard the assistance models infer through scene-grounded reasoning, beyond screening input content.
| Comments: | VLM safety, multimodal jailbreaks, and scene-state reasoning |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23206 [cs.CV] |
| (or arXiv:2611.23206v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23206 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tongxi Wu [view email]
[v1]
Tue, 17 Nov 2026 18:48:02 UTC (2,519 KB)
来源:arXiv cs.AI · arxiv.org