arXiv cs.AI· Jaime Ruiz-Serra, Tom\'a\v{s} Gaven\v{c}iak, Jobst Heitzig·· 11 小时前AI 评分31
目标揭示：信息论赋能与基于能力的人类权力之间的桥梁
Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power
AI 导读
研究揭示信息论赋能与基于能力的人类权力两种权力度量在马尔可夫博弈中的关系：由个体权力经 Fano 不等式可得下界 B(s)，赋能则从上方约束目标揭示，形成 B(s) ≤ I(G;ξ|s) ≤ I(A_h→ξ|s) ≤ 𝔈_h(s) 的链条。仅最大化赋能或目标揭示的助手都可能失效，正向保证只能来自目标达成端。基于交互日志两项统计量可计算满意度下界，用于监测用户保留的权力。
正文 · 原文
Abstract:Power matters for AI safety. Optimal agents tend to acquire power across a wide range of objectives (instrumental convergence). As we embed AI agents into socio-technical systems, humans risk gradual disempowerment. Both concerns call for a formal, quantitative measure of power. Two such measures in the context of Markov games are information-theoretic empowerment, the capacity of the channel from an agent's actions to future states, and capability-based individual power, a measure of goal attainment built from the probabilities that a human would attain each goal in a family of candidates were they pursuing it. We show where the two meet. From individual power, Fano's inequality yields a floor $B(s)$. This floor is a lower bound on what we term goal revelation, the mutual information $\mathbb{I}(G;\xi \mid s)$ between the human's latent goal and the realised trajectory. Empowerment caps revelation from above, so that $B(s) \le \mathbb{I}(G;\xi \mid s) \le \mathbb{I}(\mathbf{A}_h \to \xi \mid s) \le \mathfrak{E}_h(s)$. Each quantity can be large while the one below it stays at zero, highlighting the following failure modes. An assistant maximising empowerment can prefer an interface offering broad influence over goal-irrelevant degrees of freedom to one that lets the human attain every goal they might hold. An assistant maximising revelation cannot tell a working interface from one rewired so that every goal yields the wrong outcome, since relabelling goals leaves mutual information unchanged while zeroing every attainment probability. Positive guarantees therefore come only from the attainment end of the chain. A satisfaction floor, computable from two statistics of interaction logs, offers a way to monitor the power users retain in a deployed system, and simulations in random environments show the three quantities are correlated but far from interchangeable.
| Comments: | 19 pages, 3 tables. Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Information Theory (cs.IT); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.07247 [cs.AI] |
| (or arXiv:2611.07247v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07247 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jaime Ruiz Serra [view email]
[v1]
Wed, 4 Nov 2026 23:54:35 UTC (34 KB)
来源：arXiv cs.AI · arxiv.org