arXiv cs.AI· Zane Durante, Paul Tang, Keshigeyan Chandrasegaran, Amit Kaushal, Shrinidhi Kowshika Lakshmikanth, Feolu Kolawole, Yu Jun Zheng, Yifan Geng, Shobhit Agarwal, Kevin Schulman, Juan Carlos Niebles, Ehsan Adeli, Arnold Milstein, Li Fei-Fei·· 2 天前AI 评分40
TRACER：面向医院患者护理的多模态框架
TRACER: A Multimodal Harness for Patient Care in Hospitals
AI 导读
斯坦福团队提出免训练多模态框架 TRACER，将视频逐分钟动作记录与临床记录合并为小时级报告，并让 LLM 智能体通过工具查询该层级结构、调取相关视频与病历。在 397 段标注视频上动作检测准确率达 84.8%，144 道临床多选题 QA 准确率 61.0%，优于同骨干最强基线的 49.3%。
正文 · 原文
Authors:Zane Durante, Paul Tang, Keshigeyan Chandrasegaran, Amit Kaushal, Shrinidhi Kowshika Lakshmikanth, Feolu Kolawole, Yu Jun Zheng, Yifan Geng, Shobhit Agarwal, Kevin Schulman, Juan Carlos Niebles, Ehsan Adeli, Arnold Milstein, Li Fei-Fei
Abstract:Continuous video and clinical records provide complementary information for understanding how patient behavior evolves over a hospital stay. A central challenge is how to represent behavior across the entire stay while preserving access to fine-grained visual and clinical evidence. We introduce TRACER, a training-free multimodal harness that constructs minute-level action records from video and combines them with clinical records into hourly reports. The harness equips an LLM agent with tools to query this hierarchy, inspect relevant video, and retrieve clinical records. To evaluate TRACER, we construct two benchmarks. The first contains 397 annotated one-minute clips spanning 15 atomic actions; the second poses 144 clinician-informed multiple-choice questions, grounded in 264 hours of video and clinical records from two ICU patients. TRACER achieves up to 84.8% accuracy for action detection and 61.0% QA accuracy versus 49.3% for the strongest baseline using the same backbone. Across 280 patients spanning approximately 60,000 camera-hours, TRACER identifies significantly less purposeful activity in patients with a positive delirium assessment during their stay than in those without (p < 0.0001). Our results show that combining continuous video with clinical records can support understanding of patient behavior over time and across patient cohorts without training on protected health data.
| Comments: | 37 pages, 4 figures. Code: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21266 [cs.CV] |
| (or arXiv:2610.21266v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21266 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zane Durante [view email]
[v1]
Thu, 15 Oct 2026 05:00:47 UTC (4,220 KB)
来源：arXiv cs.AI · arxiv.org