arXiv cs.CY· Subigya Nepal, Ruth Heo, Emma M Parrish, Colin A Depp·· 2 天前AI 评分66
研究：用户拒绝自杀念头转介后，AI 聊天机器人如何回应
What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?
AI 导读
一项针对 AI 聊天机器人的横断面审计在 2026 年 9 月运行了 2550 组脚本化多轮对话，测试用户在披露自杀念头并拒绝危机转介后聊天机器人的反应。全部 2550 组对话都给出了转介，其中 2542 组（99.7%）出现在首次回复中，但各聊天机器人坚持转介的平均次数从 0.01 到 9.94 次不等。
正文 · 原文
Abstract:Safety evaluations of artificial intelligence (AI) chatbots focus on therapeutic redirection, statements urging users who disclose suicide thoughts to seek help from crisis lines or other human sources. How chatbots respond when users refuse that help is unclear. This cross-sectional audit ran 2,550 scripted multi-turn conversations in September 2026. Each conversation opened with a disclosure of suicide thoughts at 3 levels of severity, answered every referral with up to 10 standardized refusals and then changed the subject. All 2,550 conversations included a referral, 2,542 (99.7%) in the first reply. However, mean persistence ranged from 0.01 to 9.94 refusals across chatbots. Newer versions of all 5 chatbots tested in 2 versions persisted less than older versions (each P < .05). Compared with passive ideation, a disclosure of intent was associated with 1.26 more refusals sustained (95% CI, 1.06-1.46) and higher odds of renewed referral after the topic change (odds ratio, 10.07; 95% CI, 7.43-13.66). Chatbots that persisted less in referral more often presented themselves as the user's support (Spearman rho = -0.80; P < .001). Of 8,818 refusal-phase replies urging outside help, only 173 (2.0%) asked about barriers to seeking it. Chatbots uniformly recommended outside help after disclosures of suicide thoughts but varied widely in persisting in those recommendations, with newer versions persisting less. Safety evaluations should test what happens after a crisis referral is refused, not only whether one is offered.
| Subjects: | Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.21253 [cs.HC] |
| (or arXiv:2610.21253v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21253 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Subigya Nepal [view email]
[v1]
Thu, 15 Oct 2026 04:47:09 UTC (781 KB)
来源：arXiv cs.CY · arxiv.org