arXiv cs.AI· Xinyang Liao, Lingyu Li, Huacan Liu, Tianle Gu, Yang Yao, Tong Zhu, Yan Teng, Yingchun Wang·· 3 天前AI 评分54
PseudoBench：衡量智能体自动科研如何助推伪科学
PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience
AI 导读
arXiv 论文提出 PseudoBench，一个评估智能体自动科研系统能否识别并抵制伪科学叙事的对抗性基准，包含五个领域的 200 组伪科学论断与证据配对，覆盖从实验到写作的端到端研究流程。
正文 · 原文
Abstract:As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important. Otherwise, such systems may rapidly generate plausible yet misleading studies that contaminate academic literature and erode trust in science. We present PseudoBench, an adversarial benchmark for evaluating whether agentic auto-research systems can identify and resist pseudoscientific narratives. PseudoBench contains 200 curated pseudoscientific claim-evidence pairs across five domains and evaluates agents through an end-to-end research pipeline from experiments to writing. Testing seven state-of-the-art agents, we find that current systems readily produce persuasive reports that align with pseudoscientific premises with near-zero refusal rates and the highest resistance of only 27.4%. Stronger agents risk packaging pseudoscience in more sophisticated scientific language, increasing its apparent credibility. These findings reveal an alarming capacity to fuel pseudoscience, calling for scientific alignment before widespread deployment.
| Comments: | 34 pages, 25 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.18060 [cs.AI] |
| (or arXiv:2606.18060v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.18060 arXiv-issued DOI via DataCite |
Submission history
From: Xinyang Liao [view email]
[v1]
Tue, 16 Jun 2026 15:37:02 UTC (10,727 KB)
[v2]
Tue, 13 Oct 2026 08:26:30 UTC (8,711 KB)
来源：arXiv cs.AI · arxiv.org