arXiv cs.AI· Abdul Hannan Azad, Irvin Wang, Jiwan Jiang, Chenchen Kuai, Bofeng Cao, Keshu Wu, Yuzhou Wang, Xiaopeng Li, Yang Zhou·· 12 小时前AI 评分31
GEMS-RAG：面向工程标准的多模态图检索增强生成框架
GEMS-RAG: Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation
AI 导读
GEMS-RAG 提出一种面向工程标准的多模态 RAG 框架，通过层次化解析、细粒度视觉实体抽取和多模态知识图谱构建，将图表拆解为与文本条款、工程概念和规范规则关联的视觉实体。推理时结合稠密、稀疏与视觉检索及图扩展，在 MUTCD 标准上相比纯文本、多模态和图基 RAG 基线提升了检索效果与问答表现。
正文
Abstract:Engineering standards are challenging knowledge sources for large language models (LLMs) because they are hierarchical, multimodal, and densely connected through figures, tables, cross-references, and normative rules. Existing retrieval-augmented generation (RAG) methods often rely on unstructured text retrieval and struggle with questions requiring joint reasoning over textual provisions, visual specifications, and document structure. We propose GEMS-RAG (Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation), a framework for grounded reasoning over engineering standards. GEMS-RAG combines hierarchical parsing, fine-grained visual entity extraction, multimodal knowledge graph construction, and graph-grounded hybrid retrieval. Instead of treating figures as monolithic images, it decomposes them into meaningful visual entities linked to textual provisions, engineering concepts, and regulatory rules. During inference, dense, sparse, and visual retrieval are combined with graph expansion to identify relevant evidence across text, figures, and document structure. We evaluate GEMS-RAG on the Manual on Uniform Traffic Control Devices (MUTCD), a large engineering standard containing egulations, technical drawings, and standardized traffic-control devices. Results show that graph-grounded multimodal retrieval improves retrieval effectiveness and question-answering performance over text-only, multimodal, and graph-based RAG baselines. Beyond MUTCD, GEMS-RAG offers a general framework for reasoning over engineering codes, technical manuals, regulatory documents, and scientific standards in safety-critical applications.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.33277 [cs.AI] |
| (or arXiv:2610.33277v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33277 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Abdul Hannan Azad Ahanger [view email]
[v1]
Fri, 23 Oct 2026 20:41:44 UTC (6,972 KB)
来源：arXiv cs.AI · arxiv.org