arXiv cs.AI· Biao Xiang, Ali Eshragh, Yuexing Li, Kai Wang·· 9 天前AI 评分13
面向离策略评估的预算化多源反事实标注
Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation
AI 导读
研究针对上下文赌博机离策略评估（OPE）中反事实标注的预算化获取问题，在给定各来源成本与误差特征下，将标注分配建模为上下文-动作对与标注来源上的整数规划问题，以最小化依赖标注方案的估计量方差分量。
正文 · 原文
Abstract:Off-policy evaluation (OPE) estimates the value of a target policy from logged data, but limited behavior-policy coverage can force high-variance reweighting or reward-model extrapolation. Counterfactual annotations can add evidence about unobserved actions, yet practical sources, including domain experts and large language models (LLMs), may be costly, biased, or noisy. We study budgeted acquisition of such annotations for contextual-bandit OPE. Given source-specific costs and error profiles, we formulate an integer allocation problem over context-action pairs and annotation sources to minimize the component of estimator variance that depends on the annotation plan. We characterize when annotations are valuable through a first-annotation threshold and local annotation-value regimes. For the coupled multi-source problem, we develop a majorization-minimization algorithm with dynamic-programming subroutines that monotonically improves the objective. Experiments in synthetic clinical and LLM-annotated education bandits show that our allocation method reduces fixed-profile mean squared error (MSE) by 20.58% and 10.77%, respectively, relative to no annotation.
| Comments: | Accepted to NeurIPS 2026. Code available at this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC) |
| Cite as: | arXiv:2610.10974 [cs.LG] |
| (or arXiv:2610.10974v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10974 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Biao Xiang [view email]
[v1]
Wed, 7 Oct 2026 22:55:15 UTC (662 KB)
来源：arXiv cs.AI · arxiv.org