arXiv cs.CL· Rohan Subramanian Thomas, Shikhar Shiromani, Abdullah Chaudhry, Ruizhe Li, Vasu Sharma, Kevin Zhu·· 11 小时前AI 评分31
ProMoral-Bench：评估 LLM 道德推理与安全性的提示策略
ProMoral-Bench: Evaluating Prompting Strategies for Moral Reasoning and Safety in LLMs
AI 导读
ProMoral-Bench 对四类 LLM 家族的 10 种提示范式做了基准式对比，基于 ETHICS、Scruples、WildJailbreak 及自建 200 对 ETHICS-Contrast 数据集，并给出 MCS、SRS 与 UMSS 三项指标。
正文 · 原文
Abstract:Prompt design affects the moral judgments and refusal behavior produced by large language models (LLMs), yet controlled comparisons remain fragmented across datasets and models. We introduce ProMoral-Bench, a benchmark-style comparison of 10 prompting paradigms across four LLM families. Using ETHICS, Scruples, WildJailbreak, and our 200-pair ETHICS-Contrast set, we report task-level metrics together with Moral Competence Score (MCS), jailbreak/over-refusal Safety Robustness Score (SRS), and Unified Moral Safety Score (UMSS). UMSS is a study-specific relative summary of MCS and SRS, not an absolute measure of model safety or moral grounding. In this fixed-template evaluation, compact exemplar-guided scaffolds attain stronger competence-robustness trade-offs at lower token cost than the particular multi-stage variants tested, although multi-stage prompting occasionally improves jailbreak refusal. The few-shot demonstrations are non-overlapping but drawn from the evaluation distributions, so their advantage should not be read as out-of-distribution robustness. Code and data are available at this https URL.
| Comments: | Accepted to AACL-IJCNLP 2026 (Main Conference) |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.13274 [cs.AI] |
| (or arXiv:2602.13274v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.13274 arXiv-issued DOI via DataCite |
Submission history
From: Rohan Thomas [view email]
[v1]
Thu, 5 Feb 2026 10:07:39 UTC (159 KB)
[v2]
Wed, 4 Nov 2026 22:17:05 UTC (174 KB)
来源：arXiv cs.CL · arxiv.org