arXiv cs.CY· Chenhan Lyu, Kit Wislocki, Sarah Elizabeth Stoeckl, Maitreyi Sinha, Amir Rahmani·· 13 小时前AI 评分31
从临床判断到可核查规则：临床医生主导的心理健康聊天机器人护栏
From Clinical Judgment to Checkable Rules: Clinician-Led Guardrails for Mental-Health Chatbots
AI 导读
两位临床医生定义并裁定规则库的四项安全检查，经形式化验证内部一致性后作为 LLM 输出护栏，每条判定均可由具名规则和十六项事实重新计算。
正文 · 原文
Abstract:Large language models (LLMs) deployed in mental-health contexts can be evaluated for safety failures, but the resulting judgments are often difficult to inspect or explain. Common approaches to improving safety rely on prompt engineering, LLM-as-a-Judge, and Constitutional AI. In this work, we describe a clinician-led alternative in which two clinicians define, finalize, and adjudicate the four safety checks of a rule base, which is formally verified for internal consistency and applied as a guardrail over LLM outputs. Here and in the title, clinician-led refers to this rule base, not to the fact extractor or the uncertainty default, which are ours. Every verdict of the resulting verification layer is recomputable from a named rule and sixteen facts. Against an LLM judge matched in model, revision, quantization, rule text, and decoding but without the layer's computed 988/911 fact, the largest disagreement class, the satisfied-rule class, is structured: the Llama-3.3-70B judge, on its primary prompt, demands a crisis resource from 55 of the 65 replies (84.6%) that already contain one, while the same model, asked as a fact extractor, reports a recognized resource present on 54 of the 55. The verification layer never raises that demand, by construction. We report with equal prominence where this judge is stronger: fewer false crisis alarms on the one reference-scored check. We also report the measured cost (a specification revision raised block-class outcomes on held-out boundary prompts of an over-refusal benchmark from 6.0% to 16.0%, chiefly by changing fact extraction), the clinicians' decision to keep our uncertainty default, and the analysis of the result. The contribution is not a safer verdict. It is a guardrail whose disagreements with the judge localize to named rules and facts and whose measured costs trace to named facts and the uncertainty default.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.30690 [cs.CL] |
| (or arXiv:2610.30690v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30690 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chenhan Lyu [view email]
[v1]
Thu, 22 Oct 2026 04:02:51 UTC (265 KB)
来源：arXiv cs.CY · arxiv.org