arXiv cs.CL· Yuyao Zhang, Yakun Cui, Han Zhu, Jinsong Lu, Yike Guo, Sirui Han·· 13 小时前AI 评分45
FrameBench-HPS 基准揭示高风险决策中 LLM 的增益-损失框架效应
Same Probabilities, Different Advice: Gain-Loss Framing Effects in High-Stakes LLM Decision Support
AI 导读
研究提出 FrameBench-HPS 基准，包含 240 个高风险场景衍生的 2,880 条提示词，覆盖医疗、金融、教育/职业和法律/消费者权益领域。在七个 LLM 上，增益-损失框架使 32.4% 的完整组改变了推荐类别，且多数变化涉及不确定性而非直接反转。概率规范化可显著降低框架敏感性，表明概率表示是提升高风险 LLM 决策支持一致性的可行改进目标。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly used to support consequential decisions, yet their reliability under equivalent representations of uncertainty remains unclear. Existing work has documented prompt sensitivity, cognitive biases, and confidence miscalibration, but it is still unknown whether high-stakes advice changes when the same probability is expressed through complementary gain and loss frames. We introduce FrameBench-HPS, a controlled benchmark of 2,880 prompts derived from 240 high-stakes scenarios across healthcare, finance, education/career, and legal/consumer-rights domains. Across seven LLMs, gain-loss framing changes the recommendation category in 32.4% of complete groups, with most changes involving uncertainty rather than direct reversals. Human evaluation confirms corresponding shifts in perceived risk and directiveness. Robustness experiments show that framing effects persist under conversational paraphrases and exceed ordinary repeated-generation variability. Probability canonicalization substantially reduces framing sensitivity, identifying probability representation as a practical target for improving consistency in high-stakes LLM decision support.
| Comments: | 14 pages, 2 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.29294 [cs.AI] |
| (or arXiv:2610.29294v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29294 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuyao Zhang [view email]
[v1]
Wed, 21 Oct 2026 06:46:33 UTC (1,465 KB)
来源：arXiv cs.CL · arxiv.org