arXiv cs.CL· Balkrishna Giri, Md Toufique Hasan, Jussi Rasku, Muhammad Waseem, Pekka Abrahamsson·· 11 小时前AI 评分35
可信 RAG：用于检测生成式 AI 系统中虚假信息与知识投毒的评估智能体
Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems
AI 导读
研究者提出一种 Evaluation Agent 中间件，结合 NLI 事实核验、五信号投毒检测器与 Trust Index（T = 0.4F + 0.35C + 0.25(1-P)），在生成前检测被投毒的上下文。
正文 · 原文
Abstract:Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) outputs in external knowledge, but RAG systems usually trust whatever they retrieve, creating a Security-Reliability Gap: high semantic relevance does not guarantee factual truth. Adversaries exploit this through knowledge poisoning, inserting malicious documents to cause targeted misinformation. We propose an Evaluation Agent, middleware that combines Natural Language Inference (NLI) factual verification, a five-signal poison detector with relevance-weighted aggregation, and a Trust Index T = 0.4 F + 0.35 C + 0.25 (1 - P ) with a non-linear dampener for high-contamination contexts. On TruthfulQA with Llama 3.3 70B, the agent reaches 91% accuracy and 100% precision, with 100% recall on instruction injection, while in-place edits, such as entity swaps, remain hard to detect. Across three LLMs the Trust Index stays discriminative, with a Receiver Operating Characteristic Area Under the Curve (ROC-AUC) of 0.73 to 0.81; generation style matters more than model size, and per-LLM threshold calibration restores baseline competitive accuracy, whereas a weaker FEVER result shows that cross-dataset generalization requires domain-specific calibration. In a software-engineering use case, a secure-coding assistant over guidance from the Open Worldwide Application Security Project (OWASP) Top 10 and the Common Weakness Enumeration (CWE), the agent reliably blocks instruction injection of unsafe advice (F1 92%), while contradiction and subtle semantic weakening remain hard. Throughout, the agent measures detection of poisoned context before generation, not whether the LLM adopts the injected misinformation. We release the proposed approach, attack generator, and experimental artifacts at the link: this https URL.
| Comments: | 7 pages, 1 figure, 6 tables. Published in the Main Research Track of the Twenty-First International Conference on Software Engineering Advances (ICSEA 2026) |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2608.21095 [cs.SE] |
| (or arXiv:2608.21095v2 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2608.21095 arXiv-issued DOI via DataCite |
|
| Journal reference: | Published in ICSEA 2026: The Twenty-First International Conference on Software Engineering Advances, pp. 109-115, IARIA (2026) |
Submission history
From: Md Toufique Hasan [view email]
[v1]
Fri, 21 Aug 2026 13:42:41 UTC (22 KB)
[v2]
Tue, 3 Nov 2026 20:30:35 UTC (22 KB)
来源：arXiv cs.CL · arxiv.org