arXiv cs.CL· Meysam Sabbaghan, Doina Caragea·· 13 小时前AI 评分23
B-CARE：面向立场错误排序的盲式逐类相对证据评估框架
B-CARE: Blind Classwise Assessment of Relative Evidence for Stance Error Ranking
AI 导读
B-CARE 是一个预测盲的立场分类错误排序框架，仅凭目标与文本，由固定 LLM 验证器为所有候选立场标签打序数证据分，不查看基座模型的预测或置信度。在四个立场基准、三个异构基座 LLM 共 12 组设置、5,206 条测试实例上，B-CARE 平均 AUROC 为 0.735、AUPRC 为 0.524，在风险最高的 20% 预测中捕获 41.06% 的错误。
正文 · 原文
Abstract:Stance classifiers make errors that can be costly to identify under limited human-review budgets. Prioritizing potentially incorrect predictions remains challenging, particularly across heterogeneous base models. We introduce B-CARE (Blind Classwise Assessment of Relative Evidence for Stance Error Ranking), a prediction-blind framework for identifying high-risk stance predictions. Given only a target and text, a fixed LLM verifier assigns ordinal evidence scores to all candidate stance labels without observing the base model's prediction or confidence. B-CARE constructs a comparative risk score by contrasting the strongest alternative-class evidence with the evidence supporting the predicted class. The resulting classwise evidence profile can be reused across multiple base models without training a separate error detector or calibrating risk scores. We evaluate B-CARE on four stance benchmarks and three heterogeneous base LLMs, covering 12 dataset-model settings and 5,206 unique test instances. Evaluation includes error-ranking discrimination (AUROC and AUPRC), selective prediction (E-AURC), and review effectiveness (ErrorCapture at fixed review budgets). B-CARE achieves a mean AUROC of 0.735 and a mean AUPRC of 0.524, capturing 41.06% of errors within the highest-risk 20% of predictions. Although B-CARE does not uniformly outperform strong confidence-based baselines, it provides a reusable, complementary evidence signal for prioritizing potentially incorrect stance predictions across heterogeneous models.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.29350 [cs.CL] |
| (or arXiv:2610.29350v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29350 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Meysam Sabbaghan [view email]
[v1]
Wed, 21 Oct 2026 07:43:27 UTC (7,953 KB)
来源：arXiv cs.CL · arxiv.org