arXiv cs.AI· Harsh Parikh, Gabriel Levin-Konigsberg, Sohrab Andaz, Akhil Bagaria, Dominique Perrault-Joncas, Alexander Volfovsky·· 12 小时前AI 评分26
模拟器设定错误时如何收集重校准数据：探索与实验策略研究
Exploration and Experimentation with a Misspecified Simulator
AI 导读
针对校准于历史数据的模拟器在动态漂移后策略次优的问题，研究比较了四种数据收集策略：从不重校准、用 on-policy 数据被动适应、全单元持续探索，以及在预设窗口内对部分单元做随机实验。理论分析与自动售货机模拟显示，当漂移在 on-policy 数据中不可见或重校准滞后于持续漂移时，实验优于被动适应；当尝试较差动作代价高且动态可能未变时，实验优于探索。
正文 · 原文
Abstract:Randomized experiments are the standard for measuring causal effects that guide decisions, but are costly. Simulators calibrated to historical data offer a cheaper alternative, but their policies can become suboptimal under drift, a change in dynamics after calibration. We study if, when, and how to collect data for recalibrating the simulator. We consider a system with multiple units running in parallel and managed by a single operator. The real-world system is a partially observed process with a latent variable, while the simulator models it as a fully observed Markovian process, and thus is misspecified. Two data collection strategies exploit the current policy: (i) never recalibrating and (ii) recalibrating from on-policy data (passive adaptation). Two others collect off-policy data by trying actions outside the current policy: (iii) continuous exploration across all units and (iv) randomized experimentation on a sample of units over a predefined window. We investigate these strategies theoretically and in a vending-machine simulation. Experiments can improve on passive adaptation when drift is invisible in on-policy data or recalibration lags behind continuing drift. Experimentation is preferable to exploration when trying a worse action is expensive and the dynamics may not have changed.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Methodology (stat.ME) |
| Cite as: | arXiv:2605.21458 [cs.AI] |
| (or arXiv:2605.21458v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.21458 arXiv-issued DOI via DataCite |
Submission history
From: Harsh Parikh [view email]
[v1]
Wed, 20 May 2026 17:48:14 UTC (734 KB)
[v2]
Fri, 16 Oct 2026 14:56:33 UTC (265 KB)
来源：arXiv cs.AI · arxiv.org