跳到正文
arXiv cs.AI· Enshuo Hsu, Linh Nguyen, Sunyang Fu, Heath Goodrum, Sandra Stansberry, Kirk Roberts, Hongfang Liu, Yen-Chi L. Le·· 4 小时前AI 评分34

基于多模态大语言模型的扫描版认知与行为评估表视觉信息抽取系统

A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments

AI 导读

研究提出一套模块化多模态 LLM 系统,用表单分类器加对应分数抽取器处理 MMSE、MoCA、Mini-Cog 及 NICHQ Vanderbilt 扫描评估表,认知评估端到端准确率 88%,NICHQ Vanderbilt 达 97.5%。

正文 · 原文

View PDF

Abstract:Objective. Paper-based assessments are common in patient care, but handwriting and scanning artifacts make these forms difficult to process at scale. This study proposes a modular, multimodal LLM-based system with human oversight for extracting fields and scores. We conducted a qualitative analysis of the use of LLMs for hand-drawing analysis.
Materials and Methods. The system processes cognitive assessments (i.e., MMSE, MoCA, Mini-Cog) and the NICHQ Vanderbilt Assessment using two modules: 1) a form classifier that assigns a scanned page to a form type, followed by a corresponding 2) score extractor that extracts fields and scores into a schema. A human-in-the-loop design utilizing dual-model validation engages human annotators when two LLMs disagree, limiting manual review to challenging pages only.
Results. End-to-end accuracy for cognitive assessments was 88%, with most form types exceeding 90%. For the NICHQ Vanderbilt, overall accuracy reached 97.5%, with all form types above 96%. The human-in-the-loop design further improved cognitive assessment accuracy to 97.5% while reducing manual work by 63.8%. In the Mini-Cog Clock Drawing task, qualitative analysis showed that 62.3% of all extraction errors stemmed from ambiguous drawing, in which cases the LLM provided reasonable readings.
Discussion. Compared to current manual data extraction, our LLM-based system reduces processing time by 10 to 20 times and costs by over 12 times. The modular design allows seamless expansion to additional form types.
Conclusion. Our system provides an accurate, scalable, and cost-efficient solution for structuring data trapped in handwritten paper forms for clinical decision support.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22252 [cs.AI]
  (or arXiv:2611.22252v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.22252

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Enshuo Hsu [view email]
[v1] Tue, 17 Nov 2026 05:56:48 UTC (2,200 KB)

来源:arXiv cs.AI · arxiv.org