arXiv cs.AI· Rushab Munot·· 13 小时前AI 评分31
多选皆正确时：Jev 等决策模型的非对称概率分配研究
When Multiple Choices Are Correct: Asymmetric Probability Allocation in Jev and Other Decision Models
AI 导读
研究评估了 Jev、Liquid d1 和 GPT-6 Luna Decisions 三个决策模型在 MetaTool、HotpotQA 和 ECtHR 三个数据集上的概率分配与阈值表现。
正文 · 原文
Abstract:Decision models return normalized probability distributions over candidate choices, which downstream applications can use to select among choices. Although their providers emphasize the reliability of these probabilities, calibration on tasks with a single correct choice does not establish their reliability when multiple choices are correct. We evaluate three decision models (Jev, Liquid d1, and GPT-6 Luna Decisions) on three diverse datasets (MetaTool, HotpotQA, and ECtHR) to study probability allocation and thresholding. Even when models rank every correct choice above every distractor, in 28% to 50% of those lists, at least one correct choice receives a probability below the uniform probability of 1/n, where n is the number of choices. We also present each model with five identical copies of a correct choice and observe probability disparities, with mean gaps ranging from 17 to 79 percentage points. We further show how these allocation disparities affect the recovery of multiple correct choices under probability thresholding. Decision-model probabilities can rank correct choices, but a probability threshold does not identify the set of correct choices.
| Comments: | 11 pages, 2 figures. Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.29217 [cs.AI] |
| (or arXiv:2610.29217v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29217 arXiv-issued DOI via DataCite |
Submission history
From: Rushab Munot [view email]
[v1]
Wed, 21 Oct 2026 05:18:35 UTC (92 KB)
来源：arXiv cs.AI · arxiv.org