arXiv cs.CL· Lin Ai, Emma Thronson, Katherine Conhaim, Amir Zeldes·· 3 天前AI 评分20
PROPALIGN：基于多摘要的命题显著性评分评估方法
The Sum of All Summaries: PROPALIGN for Graded Proposition Salience Evaluation
AI 导读
研究者提出 PROPALIGN，一种基于多摘要的命题显著性评分估计方法，并发布配套数据集 PROPALIGNBANK，包含超 35K 条命题的 138K 对齐数据。在英语口语与书面语多体裁对比中，直接让模型判断会导致不一致，而基于摘要的分级指标更稳定；单模型表现弱于人类，但融合语言特征的模型集成可接近人工标注质量上限。
正文 · 原文
Abstract:Identification and ranking of salient propositions in documents is indirectly involved in many AI tasks, but few papers define proposition salience scores clearly, make diverse data available, or evaluate models directly on the task. This paper proposes PROPALIGN, a multiple-summary-based approach for propositional salience score estimation, and releases PROPALIGNBANK, an accompanying dataset for the method with 138K alignments of over 35K propositions. We compare several approaches to salient proposition extraction in English across a broad range of spoken and written genres, and demonstrate that naively asking models for judgments leads to inconsistencies. By contrast, graded, summary-based metrics are more stable. We further observe that while single models perform worse than humans, an ensemble of models incorporating linguistic features approaches manual annotation quality ceilings. In addition, analyses of the model outputs suggest insights into how models can extract salient information with and without the use of summaries.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19387 [cs.CL] |
| (or arXiv:2610.19387v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19387 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lin Ai [view email]
[v1]
Wed, 14 Oct 2026 01:32:24 UTC (423 KB)
来源：arXiv cs.CL · arxiv.org