arXiv cs.AI· Ilgin Dogan, Zuo-Jun Max Shen, Anil Aswani·· 11 小时前AI 评分18
面向完美知识智能体的无观测奖励激励策略学习
Learning to Incentivize Perfect-Knowledge Agents with Unobserved Rewards
AI 导读
研究多臂老虎机框架下的重复委托-代理博弈：委托方为每个臂设置不同激励，智能体选择使自身期望奖励加激励最大化的臂，而委托方无法直接观测智能体获得的奖励。作者先构造智能体各臂期望奖励的估计器（可视为 MAB 中的在线逆优化），再据此设计策略，并证明在智能体完全知晓自身期望奖励时达到低遗憾（至多对数因子的平方根遗憾）。数值模拟展示了该框架在长期他汀类药物治疗的个性化依从性激励设计中的应用。
正文 · 原文
Abstract:Motivated by a number of real-world applications from domains like healthcare and sustainable transportation, in this paper we study a scenario of repeated principal-agent games within a multi-armed bandit (MAB) framework, where: the principal gives a different incentive for each bandit arm, the agent picks a bandit arm to maximize its own expected reward plus incentive, and the principal observes which arm is chosen and receives a reward (different than that of the agent) for the chosen arm. Designing policies for the principal is challenging because the principal cannot directly observe the reward that the agent receives for their chosen actions, and so the principal cannot directly learn the expected reward using existing estimation techniques. As a result, the problem of designing policies for this scenario, as well as similar ones, remains mostly unexplored. In this paper, we construct a policy that achieves a low regret (i.e., square-root regret up to a log factor) in this scenario for the case where the agent has perfect-knowledge about its own expected rewards for each bandit arm. We design our policy by first constructing an estimator for the agent's expected reward for each bandit arm. Since our estimator uses as data the sequence of incentives offered and subsequently chosen arms, the principal's estimation can be regarded as an analogy of online inverse optimization in MAB's. Next, we construct a policy that we prove achieves a low regret by deriving finite-sample concentration bounds for our estimator. We conclude with numerical simulations demonstrating the applicability of our framework to personalized adherence-improving incentive design for long-term statin treatments.
| Comments: | 61 pages, 3 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT); Machine Learning (stat.ML) |
| Cite as: | arXiv:2304.07407 [cs.LG] |
| (or arXiv:2304.07407v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2304.07407 arXiv-issued DOI via DataCite |
Submission history
From: Ilgin Dogan [view email]
[v1]
Fri, 14 Apr 2023 21:57:16 UTC (299 KB)
[v2]
Sun, 7 May 2023 19:30:01 UTC (300 KB)
[v3]
Wed, 4 Nov 2026 08:24:56 UTC (263 KB)
来源：arXiv cs.AI · arxiv.org