arXiv cs.AI· Jiawen Lu, Tongtong Wu·· 10 天前AI 评分22
类型化决策模型中候选覆盖与拒绝策略迁移的基准测试
Benchmarking candidate coverage and rejection policy transfer in typed decision models
AI 导读
研究对比 Laya、Jev 与 Qwen2.5-7B-Instruct 在候选集和任务变化下的拒绝策略迁移，发现源域校准常无法保持目标工作点：在 DBpedia 上校准的 Jev 策略会拒绝 69.3% 的 Emotion 测试输入，而 Emotion 策略则完全失去检测能力。
正文 · 原文
Abstract:Rejection policies must remain useful as candidate sets and tasks change. We compare Laya, Jev and Qwen2.5-7B-Instruct using public reference labels, testing Laya/Jev policy transfer at equal calibration budgets and all three models on artificial omission, natural retrieval misses and public out-of-scope queries. Source calibration often fails to preserve the target operating point. A Jev policy calibrated on DBpedia rejects 69.3% of covered Emotion test inputs, while an Emotion policy loses detection entirely. Retrieval exposes a different tradeoff: with ten intent candidates, Laya detects 99.0% of out-of-scope queries but rejects 48.8% of covered queries. Separating missing-answer sources reveals these costs alongside retrieval coverage. The benchmark provides shared inputs, explicit decision and failure categories, and reproducible scoring to assess rejection policies under the conditions in which they are reused. Code and benchmark artifacts are available at this https URL.
| Comments: | 29 pages, 5 figures. v2: expanded evaluation with Qwen2.5-7B-Instruct and CLINC150; added fixed-budget rejection policy transfer, three missing-answer sources, and controlled robustness analyses. Code and benchmark artifacts: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.03387 [cs.AI] |
| (or arXiv:2610.03387v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03387 arXiv-issued DOI via DataCite |
Submission history
From: Jiawen Lu [view email]
[v1]
Fri, 2 Oct 2026 14:36:46 UTC (64 KB)
[v2]
Tue, 6 Oct 2026 02:17:37 UTC (248 KB)
来源：arXiv cs.AI · arxiv.org