arXiv cs.AI· Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Rui Chen, Daren Zha·· 13 小时前AI 评分22
OSCC：面向不完美信息学习的观测安全耦合优化框架，用于梯度噪声控制
OSCC: Certified Observation-Safe Coupling Optimization for Gradient-Noise Control in Imperfect-Information Learning
AI 导读
研究者提出观测安全反事实耦合框架 OSCC，通过边际保持、信息状态安全等五项条件定义可接受的耦合类，并推导出策略梯度噪声变化由策略雅可比加权的非对角收益协方差决定。
正文 · 原文
Abstract:Coupled rollouts can reduce the noise of counterfactual action comparisons, but two issues prevent standard common-random-number constructions from serving as a general learning primitive in imperfect-information environments. First, an invalid coupling may expose hidden state, synchronize endogenous policy randomness, or misalign chance events after counterfactual histories diverge. Second, in multi-action policy optimization, lower return-contrast variance is not by itself the relevant objective: the optimizer depends on the return covariance matrix after projection through the local policy-gradient geometry. We introduce observation-safe counterfactual coupling (OSCC), a framework that defines an admissible class through marginal preservation, information-state safety, branch-local policy randomness, semantic event alignment, and trace-before-oracle replay. We derive a gradient-aware coupling criterion showing that, for marginal-preserving couplings, policy-gradient noise changes are determined by policy-Jacobian-weighted off-diagonal return covariance. This motivates OSCC-Select, a calibration-only selector that chooses among independent, root-only, continuation-only, and fully coupled rollouts using separate safety and gain certificates. Its gain target combines projected gradient noise with measured physical sampling cost and falls back to independent sampling whenever a simultaneous lower confidence bound does not certify improvement. On 100,000 fixed-root Leduc comparisons, the fully coupled CP-GRPO instantiation reduces return-contrast variance from 41.1158 to 18.1441, a 55.87% reduction, while preserving the declared branch marginals. With three actions, OSCC-Select chooses continuation coupling and attains gradient-noise trace 0.0783 versus 0.0917 for return-variance selection. Increasing calibration from 64 to 2,048 groups raises certification from 0.327 to 0.995.
| Comments: | 34 pages, 10 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.33543 [cs.AI] |
| (or arXiv:2609.33543v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.33543 arXiv-issued DOI via DataCite |
Submission history
From: Miaobo Hu [view email]
[v1]
Sun, 27 Sep 2026 13:10:30 UTC (1,712 KB)
[v2]
Fri, 16 Oct 2026 10:56:55 UTC (1,712 KB)
来源：arXiv cs.AI · arxiv.org