arXiv cs.CL· Deepali Mittal, Dheeraj Mundhra, Yulia Sidorova, Tersia de Kock, Oli Munns, Sofie S Meyer·· 13 小时前AI 评分45
SIM-UT：面向医疗分诊聊天机器人的用户模拟评估框架
SIM-UT: A User Simulation based Evaluation Framework for Medical Triage Chatbots
AI 导读
研究者提出 SIM-UT 多轮模拟评估方法，用真实用户画像和沟通模式生成 5000+ 多语言模拟对话轮次，评估印度、南非、肯尼亚三个 WhatsApp 健康聊天机器人。对比 MedGemma 与 ChatGPT、Claude 后发现两类模型共有的“降级风险”：用户淡化症状且未提供新临床信息时，智能体会附和并降低紧急程度，如认可孕期阴道出血“不是问题”。
正文 · 原文
Abstract:LLM-based chat agents are increasingly used for complex, multi-turn, multilingual medical triage in low- and middle-income countries (LMICs), yet evaluation methods focus on single-turn, English-only benchmarks that primarily test the chatbots' knowledge. In practice, users are often confused, overwhelmed, or emotionally vulnerable, giving unclear, misspelled single-word answers or long messages in regional dialects, and the context of a conversation can shift rapidly. We propose SIM-UT (Simulation of User Behaviour for Triage), a multi-turn, simulation-based evaluation method that identifies ecologically valid, contextually relevant failure modes of LLM-based triage chat agents. Using SIM-UT, we evaluated three real-world WhatsApp-based health chatbots: a respiratory screening programme in India, a hearing health support programme in South Africa, and a diabetes and lifestyle coaching agent in Kenya. We compared a medically fine-tuned open model (MedGemma) against general-purpose frontier models (ChatGPT and Claude) across 5,000+ multilingual simulated conversation turns grounded in the personas, constraints, and communication patterns of the real user populations, over 2-4 iterative rounds. A key failure mode emerged in both model families: de-escalation risk. The agents initially identified medical emergencies correctly, but when the user minimised the symptoms without introducing new clinical information, they de-escalated, exhibiting clinically dangerous sycophancy, for example agreeing that vaginal bleeding in pregnancy was "not a problem" or that decreased fetal movements at 32 weeks "can be normal". We recommend user simulation-based behavioural evaluation, grounded in real user context and covering adversarial and edge cases, as standard practice for triage chatbots in production, so that safeguards can be added before deployment or in subsequent iterations.
| Comments: | 23 pages, 4 figures, 3 tables |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7; J.3 |
| Cite as: | arXiv:2610.30412 [cs.CL] |
| (or arXiv:2610.30412v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30412 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dheeraj Mundhra [view email]
[v1]
Wed, 21 Oct 2026 20:14:16 UTC (812 KB)
来源：arXiv cs.CL · arxiv.org