跳到正文
arXiv cs.AI· Ting Yang, Zijing Hong, Chengjie Wang, Peng Wei, Chunxiao Guo, Yun Yue, Xurui Li, Jinqun Guan, Zixiao Chen, Ant Group·· 4 小时前AI 评分31

HealthCLBench:用受控病例簇评估健康对话中的上下文学习

HealthCLBench:Evaluating Context Learning in Health Conversations through Controlled Case Clusters

AI 导读

HealthCLBench 是面向中文健康对话的上下文学习基准,包含 328 个受控病例簇、1,103 个病例和 4,931 条评分标准,覆盖医疗咨询与日常健康管理。

正文 · 原文

View PDF HTML (experimental)

Abstract:Complex health conversations require answers grounded in individual information and currently valid evidence. We evaluate health context learning: acquiring, integrating, and applying relevant context during inference with fixed model parameters to answer health questions and respond appropriately to changes in facts, evidence, or task goals. To our knowledge, Chinese-language medical benchmarks have yet to systematically combine explicit contextual difficulties, controlled case variations, and member-specific medical criteria for open-ended answers. We introduce HealthCLBench, comprising 328 controlled clusters, 1,103 cases, and 4,931 rubric criteria across medical consultation and everyday health management. Members share backgrounds while evidence, patient states, or task goals change; each has its own medical scoring requirements. A 70-person medical team annotates and reviews the simulated cases and their rubrics. Human review of 700 full-context responses from the original evaluation yields 90.5% pooled agreement with the original automated judgments at the rubric-item level. We report full-context and query-only results for seven general-purpose models and Ant Afu. Across the eight systems, full- query differences range from 11.83 to 31.76 case points and from 10.87 to 38.97 geometric cluster points. GPT-6-Astra leads both full-context metrics at 75.03 and 60.42, followed by Claude-Opus-5-5 and Ant Afu. Cross-condition differences describe score changes under the reported evaluation settings. Response comparisons reveal failures in checking historical information, expressing uncertainty when warranted, and ensuring recommendations respect required conditions. The resource supports model comparison and targeted follow-up evaluation.
Comments: 33 pages, 7 figures, 18 Tables
Subjects: Artificial Intelligence (cs.AI)
MSC classes: 68T05, 68T50
ACM classes: I.2.7; I.2.1
Cite as: arXiv:2611.22362 [cs.AI]
  (or arXiv:2611.22362v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.22362

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ting Yang Tommy [view email]
[v1] Tue, 17 Nov 2026 08:18:04 UTC (1,744 KB)

来源:arXiv cs.AI · arxiv.org