arXiv cs.CL· James P. Beno·· 11 小时前AI 评分45
决策模型作为编码智能体的策略门控：显式策略边界提升检测并减少误报
Decision Models as Policy Gates for Coding Agents: Explicit Policy Boundaries Improve Detection and Reduce False Alarms
AI 导读
研究将22个托管决策模型和两个提示式通用模型用作多规则策略引擎的分类器，在2,643条工具调用的合成数据集上按28条规则盲标评测。以示例与反例写出显式边界后，全部23个可用模型的召回率提升5.5-26.1个百分点，21个模型误报下降；Jev 1.14召回率从89.3%升至98.2%，252个干净操作中的不必要中断从30.6%降至4.8%。
正文 · 原文
Abstract:Coding agents can destroy data, leak secrets and message the wrong person faster than a human can review. Could a decision model catch these risks at runtime? We evaluated 22 hosted decision models and two prompted general models as classifiers for a multi-rule policy engine. A synthetic dataset of 2,643 tool calls was labeled blind against 28 rules, with a sealed holdout. With a single shared preamble in both conditions, explicit boundaries written as examples and counterexamples raised recall by 5.5-26.1 percentage points on all 23 available models and reduced false positives on 21. Jev 1.14's recall rose from 89.3% to 98.2%, while unnecessary interruptions fell from 30.6% to 4.8% of 252 clean actions. With examples, its median latency was 120 ms at $0.46 per thousand requests. Strong models retained the recall gains when concrete names were replaced by generic descriptions; three largely retained them when the same boundaries were written as prose. On 16 new scenarios written after the initial studies, a follow-up improved recall on 22 of 23 models and reduced unnecessary interruptions on 21. We release the dataset, rule texts and harness.
| Comments: | 15 pages, 2 figures. Dataset, rule texts and harness available at this https URL |
| Subjects: | Computation and Language (cs.CL); Cryptography and Security (cs.CR); Software Engineering (cs.SE) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2611.02155 [cs.CL] |
| (or arXiv:2611.02155v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02155 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: James Beno [view email]
[v1]
Sun, 1 Nov 2026 19:40:30 UTC (56 KB)
来源：arXiv cs.CL · arxiv.org