arXiv cs.AI· Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia, Xinyu Liu, Alexander Heckett, Jiayuan Liu, Eden Hartman, Vojt\v{e}ch Kova\v{r}\'{i}k, Sven Neth, Maria-Florina Balcan, Tuomas Sandholm, Vincent Conitzer·· 12 小时前AI 评分34
AI 安全中的根本不可能性结果：在多项期望目标之间权衡
Fundamental Impossibility Results in AI Safety: Trading Off between Desiderata
AI 导读
研究将 AI 系统与用户的交互形式化为三方博弈：防御方与攻击方受计算或信息限制，而提供公开建议的 AI 约束更少。主要结果表明，在建议保持公开的前提下，帮助防御方存在根本性权衡。作者还考察了 AI 共享的信息能否被操纵以仅有利于防御方，并讨论了如何用这些结果设计更安全的 AI 系统。
正文 · 原文
Authors:Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia, Xinyu Liu, Alexander Heckett, Jiayuan Liu, Eden Hartman, Vojtěch Kovařík, Sven Neth, Maria-Florina Balcan, Tuomas Sandholm, Vincent Conitzer
Abstract:Is there a fundamental tradeoff between desiderata for safe AI systems, such as harmlessness and helpfulness? We answer by formalizing the interaction between AI systems and users as a game with three players: a defender and an attacker that are computationally or informationally bounded, and an AI that provides public advice and is less constrained. Our main result shows that there is a fundamental tradeoff in providing advice that helps the defender while advice remains public. We also consider whether the information that the AI shares can be manipulated to benefit only the defender and conclude with a discussion of how we can use these results in designing safer AI systems.
| Subjects: | Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.27373 [cs.GT] |
| (or arXiv:2610.27373v1 [cs.GT] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27373 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ida Mattsson [view email]
[v1]
Mon, 19 Oct 2026 22:57:06 UTC (46 KB)
来源：arXiv cs.AI · arxiv.org