arXiv cs.AI· Barbara Kitchenham, Sebasti\'an Pizard, Lech Madeyski, Ronnie de Souza Santos, Martin Shepperd, David Budgen·· 14 小时前AI 评分31
GUEST：用 GenAI 与 LLM 支持系统文献综述的使用与评估初步指南
GUEST: Preliminary Guidelines for Using and Evaluating GenAI Tools to Support Systematic Literature Reviews
AI 导读
研究团队提出 GUEST（GenAI Use and Evaluation in SLR Tasks）框架，包含 51 条建议，用于指导规划、执行和透明报告 GenAI 辅助的系统文献综述及验证研究。
正文 · 原文
Abstract:Context: Generative AI (GenAI) and Large Language Models (LLMs) are increasingly used for academic tasks in software engineering and beyond, including systematic literature reviews (SLRs). However, whilst their capabilities are rapidly developing, there is no guarantee they can meet the rigour, reliability, and transparency that SLRs require. Objectives: To support researchers intending to conduct SLRs using GenAI, or those conducting empirical studies to evaluate how well GenAI supports SLR tasks. Methods: First, we conducted a rapid review to identify studies that propose guidelines for evaluating and using GenAI and LLMs to support SLRs. Second, we drew on thought experiments, relevant guidance from the literature, and our own experience from conducting SLRs and evaluating tools, to develop recommendations for how to use and assess GenAI in the context of SLRs. Results: We classify methodological issues for the roles of evaluation and reviewing into four categories spanning SLR practice adjustments, general secondary-study standards, GenAI evaluation practices and core empirical rigour. Building on this, we introduce GUEST (GenAI Use and Evaluation in SLR Tasks) which is a comprehensive framework of 51 recommendations designed to guide researchers through planning, executing and transparently reporting GenAI-assisted literature reviews and validation studies. Conclusion: Current empirical evidence indicates that GenAI cannot be relied upon to perform unsupervised systematic reviews so requires human oversight. Nevertheless, it can offer cost-effective assistance for repetitive screening tasks and secondary validation. By standardizing how these tools are deployed and evaluated, the GUEST framework provides a necessary methodological baseline for trustworthy GenAI-assisted research in software engineering.
| Comments: | v2: revised version. 49 pages, 2 figures, 15 tables |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| ACM classes: | D.2; I.2 |
| Cite as: | arXiv:2607.24991 [cs.SE] |
| (or arXiv:2607.24991v2 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2607.24991 arXiv-issued DOI via DataCite |
Submission history
From: Lech Madeyski [view email]
[v1]
Mon, 27 Jul 2026 18:43:35 UTC (80 KB)
[v2]
Thu, 29 Oct 2026 16:42:58 UTC (91 KB)
来源：arXiv cs.AI · arxiv.org