arXiv cs.AI· Itai Zilberstein, Ioannis Anagnostides, Zachary W Sollie, Arman Kilic, Tuomas Sandholm·· 6 天前AI 评分29
面向策略优化的偏好引出方法及其在心脏移植与人类价值对齐中的应用
Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Values
AI 导读
研究者提出一种针对线性效用的偏好引出算法，直接对分配结果而非算法决策进行成对比较，分两阶段：先通过切割平面快速收缩属性权重空间并为第二阶段热启动，第二阶段可证明收敛到用户效用函数。在心脏移植分配中，该算法学到的线性效用比基于 LLM 的方法更准确，据此优化的政策与人类价值对齐度接近最优，竞争比为 0.95，而现状政策仅为 0.66。
正文 · 原文
Abstract:Preference elicitation is essential for aligning AI systems with human values. Prior approaches (e.g., for organ allocation) often ask stakeholders to compare the decisions of an algorithm (e.g., patient A vs. patient B). Such a decision-level approach conflates the means with the ends. Instead, we elicit preferences directly over allocation outcomes to learn a utility function for policy optimization. We construct a novel preference elicitation algorithm for linear utilities that outperforms prior techniques in practice. Our algorithm has two phases. The first phase learns cutting planes through pairwise comparisons to rapidly shrink the space of possible attribute weights and warm-starts the second phase by eliminating dominated regions. The second phase then provably converges to the user's utility function. We apply our technique to heart transplant allocation, where a policy must balance competing objectives such as post-transplant outcomes, waitlist mortality, geographic ease, and equity. Using our algorithm, we conduct a user study to learn and aggregate a human-value-aligned utility function. We evaluate the linear representation against LLM-based methods supplied with users' natural language explanations, finding that none of these methods is more accurate than the learned linear utilities. We use the resulting utility function to optimize heart transplant policies that are significantly better aligned with human values. Compared to the hindsight optimum, the status quo policy achieves a competitive ratio of just 0.66, while our method is near-optimal with a competitive ratio of 0.95.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.28620 [cs.AI] |
| (or arXiv:2608.28620v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.28620 arXiv-issued DOI via DataCite |
Submission history
From: Itai Zilberstein [view email]
[v1]
Wed, 29 Jul 2026 13:33:10 UTC (208 KB)
[v2]
Fri, 9 Oct 2026 13:18:06 UTC (214 KB)
来源：arXiv cs.AI · arxiv.org