arXiv cs.CL· Nadun Rajapaksha, Sandun Herath, Devnith Wijesinghe, Isiri Amani Withanawasam, Hasindri Watawana, Ranga Rodrigo·· 13 小时前AI 评分29
PathKGen：病理知识图谱辅助 MLLM，实现可解释的组织病理学报告生成
PathKGen: Pathology Knowledge Assisted MLLM for Explainable Histopathology
AI 导读
PathKGen 是知识图谱辅助的 MLLM，用于组织病理学报告生成，在 WSI-Bench 上超越此前 SOTA，并将诊断推理呈现为可视化图谱。它包含视觉路径与知识路径两条分支：前者用病理基础模型学习视觉 token 输入，后者通过查询病理专用知识图谱提取 WSI 关键形态特征以生成知识 token。该知识路径采用模块化设计，可与视觉对齐的 MLLM 互补，并通过构建领域知识图谱扩展到其他应用。
正文 · 原文
Abstract:Whole Slide Image (WSI) analysis serves as the fundamental method for the clinical diagnosis of cancer in digital histopathology. Recent advances in Multimodal Large Language Models (MLLMs) have enabled state-of-the-art (SOTA) automated histopathology report generation. However, MLLMs reason purely through distributions encoded in their neural parameters, limiting the amount of knowledge and reasoning logic they can contain given the model scale. Their black-box reasoning poses another challenge to the human interpretation of intermediate reasoning. We address these issues by utilizing knowledge graphs to inject external structured knowledge for MLLMs to reason over, while making this reasoning component visible. Knowledge graphs are capable of partially externalizing the containment of medical knowledge, an area that remains underexplored in pathology report generation. We introduce PathKGen, a knowledge graph-assisted MLLM for histopathology report generation. PathKGen has two main branches: (1) a visual path that learns visual token inputs to the MLLM using pathology foundation models, and (2) a knowledge path that learns knowledge token inputs by extracting key morphological features of WSIs through querying a pathology-specific knowledge graph. A comprehensive evaluation on WSI-Bench dataset shows that PathKGen surpasses previous SOTA methods while revealing diagnostic reasoning as visually intuitive graphs. Due to its modular design, the proposed knowledge path is complementary to vision-aligned MLLMs and can be extended to other applications by curating domain-specific knowledge graphs to generate knowledge tokens.
| Comments: | Accepted to the WACV 2027. Source code is available at this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.33691 [cs.CV] |
| (or arXiv:2610.33691v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33691 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sandun Herath [view email]
[v1]
Sat, 24 Oct 2026 14:13:13 UTC (11,631 KB)
来源：arXiv cs.CL · arxiv.org