跳到正文
arXiv cs.AI· Risha Parveen, Najmeh Akbari, Farnaz Delirie, Zheying Zhang, Fran\c{c}ois Christophe, Jussi Rasku, Pekka Abrahamsson·· 10 小时前AI 评分18

Ask the Peers:面向需求评审的 NFR 覆盖缺口排序

Ask the Peers: Ranking NFR Coverage Gaps for Requirements Review

AI 导读

研究将 NFR 覆盖缺口评审建模为项目级排序任务,对 11 个 PROMISE NFR 类别排序供分析师检查,并比较七种统计、嵌入向量与 LLM 配置。在完全移除某类别的受控缺口注入下,四种使用同类项目类别统计的配置表现接近(平均 AUROC 0.875-0.889),将统计提供给直接 LLM 排序器带来最大证据增益。所有配置在类别存在但覆盖不足时均变弱,且没有单一配置在所有 NFR 类别上最优。

正文 · 原文

View PDF HTML (experimental)

Abstract:Missing or underrepresented non-functional requirements (NFR) categories leave little direct evidence in a requirements set, making them difficult to identify during requirements review. In this study, we frame NFR coverage-gap review as a project-level ranking task: given a target requirements specification, the goal is to rank the 11 PROMISE NFR categories for analyst inspection. We compare seven statistical, embedding-based, and large language model (LLM)-based configurations that differ in both ranking mechanism and evidence source, including target requirement statements, peer requirements, their semantic representation, and peer category statistics. We evaluate the configurations using controlled gap injection with leave-one-project-out evaluation on PROMISE, PROMISE+, PURE-Transport, and PURE-Business. Four configurations that use peer category statistics are effectively tied under complete removal (0.875-0.889 mean AUROC), despite different ranking mechanisms, and supplying those statistics to a direct LLM ranker produces the largest evidence gain. All configurations weaken when a category is present but underrepresented, and none is best for every NFR category. For count-changing category-level gaps, structured peer evidence matters more than ranker sophistication. The findings suggest that the rankings should guide analyst inspection rather than serve as automatic judgements of requirements completeness.
Comments: Pre-review version of a paper subsequently accepted for publication at PROFES 2026
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23296 [cs.SE]
  (or arXiv:2611.23296v1 [cs.SE] for this version)
  https://doi.org/10.48550/arXiv.2611.23296

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Risha Parveen [view email]
[v1] Tue, 3 Nov 2026 09:55:21 UTC (1,048 KB)

来源:arXiv cs.AI · arxiv.org