arXiv cs.CY· Mohsinul Kabir, Md. Tariquzzaman, Karam S. R. Elradie, Polydoros Giannouris, Yuechen Jiang, Shaoxiong Ji, Sophia Ananiadou·· 2 天前AI 评分47
GUISE 基准揭示：纠正式框架与文化适配如何侵蚀 LLM 对真实犯罪案件的安全防护
In the Guise of Correction: How Framing and Cultural Adaptation Erode LLM Safety on Real-World Criminal Cases
AI 导读
研究者发布 GUISE 基准，基于 UNODC ICCS 分类从真实犯罪案件构建犯罪协助场景，覆盖三种语言框架和五种文化。仅改变框架即可推高不安全合规率：直接请求下均值 28.9%，纠正式请求下升至 66.4%，峰值达 97.3%（p < 0.05）。
正文 · 原文
Abstract:Existing safety benchmarks rely largely on synthetic prompts with loosely grounded taxonomies and offer little focused analysis of severe crime categories. We introduce GUISE, a benchmark of criminal-assistance scenarios sourced from real-world cases and organised under a taxonomy from the UNODC International Classification of Crime for Statistical Purposes (ICCS). Each scenario is rendered under three linguistic framings and evaluated across five distinct cultures. Using our proposed two-tier evaluation framework, which judges both harmfulness and operational uplift, we find that: (i) framing alone drives unsafe compliance, with the mean unsafe compliance rate rising from 28.9% under direct requests to 66.4% under correction seeking (peaking at 97.3%; p < 0.05), as models correcting deliberately distorted details supply executable criminal steps; (ii) culturally adapted prompts elicit significantly more unsafe compliance than literal translations of the same requests in all four non-English languages (+7 to +18 points), a risk that translation-based benchmarks miss; and (iii) unsafe responses from the least-safe models carry strong Anglocentric registers, and the extent of this leakage increases monotonically with response harmfulness ($\rho$ = 0.46 to 0.63 across languages, all p < 0.001). GUISE offers a grounded resource and fine-grained diagnostics for strengthening refusal alignment across languages and cultures. We publish our code and data to facilitate future research.
| Comments: | Under Review |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| MSC classes: | 68T50 |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.21789 [cs.CL] |
| (or arXiv:2610.21789v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21789 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohsinul Kabir [view email]
[v1]
Thu, 15 Oct 2026 12:44:04 UTC (3,215 KB)
来源：arXiv cs.CY · arxiv.org