arXiv cs.CL· Kerri Prinos, Lilianne Brush, Cameron Denton·· 12 小时前AI 评分42
Honeyquest for LLMs：AI 攻击者的网络欺骗需要重新思考
Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers
AI 导读
研究团队将 Honeyquest 评测工具改造为自动化框架，测试了来自 10 家提供商的 21 个 LLM（参数规模 8B 至超 1T）在 174 条侦察查询上的表现，共获得 10,962 条回复并与 47 人人类基线对比。
正文 · 原文
Abstract:The empirical foundation of cyber deception relies on human-centered hypotheses, but the rapid emergence of autonomous, AI-enabled attackers challenges whether this foundation transfers to AI agents. To address this, we introduce an automated evaluation framework adapted from the Honeyquest instrument to assess LLM attacker judgment at scale. Our 21-LLM cohort spanned 10 providers, diverse architectures and specializations, open- and closed-weight models, and parameter scales from 8B to over 1T. We evaluated the performance of this LLM cohort (yielding 10,962 responses) against the 47-participant human baseline across an identical set of 174 reconnaissance queries. Our empirical evaluation reveals three key findings that establish LLMs as a distinct attacker class: (1) every model in our cohort falls for deceptive traps at a significantly higher rate than human attackers; (2) the defensive attention-diversion effect observed in humans is statistically absent in our LLM cohort; and (3) a critical recognition-action gap, where LLMs successfully articulate trap recognition in their reasoning but exploit the deceptive elements anyway 73.4% of the time. Across the 21 models, trap recognition in reasoning text did not predict fell-for-trap behavior (Spearman r = +0.08, p = 0.73). Ultimately, these findings demonstrate that human-centered deception hypotheses do not reliably transfer to AI attackers, highlighting the critical need for new research into AI-native active defense frameworks.
| Comments: | 20 pages, 4 figures, 2 tables |
| Subjects: | Cryptography and Security (cs.CR); Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.21037 [cs.CR] |
| (or arXiv:2606.21037v3 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2606.21037 arXiv-issued DOI via DataCite |
Submission history
From: Lilianne Brush [view email]
[v1]
Fri, 19 Jun 2026 02:01:50 UTC (85 KB)
[v2]
Mon, 17 Aug 2026 15:13:20 UTC (87 KB)
[v3]
Fri, 23 Oct 2026 15:10:14 UTC (79 KB)
来源：arXiv cs.CL · arxiv.org