arXiv cs.AI· Yijie Hu, Wei Li, Junjie Zhou, Zhiqi Jiang, Yi Jin, Licong Xu, Xinkai Sun, Kai Duan, Lisha Zhang·· 12 小时前AI 评分38
PatentAgentBench：用专业专利任务套件评测 LLM 智能体
PatentAgentBench: Benchmarking LLM Agents with Professional Patent Task Suites
AI 导读
研究者提出 PatentAgentBench，包含 510 个实例、覆盖新颖性检索、FTO 检索、设计 FTO、无效检索、权利要求对照表、专利撰写、审查意见答复和专利翻译八类任务套件，检索任务不提供候选文档池。该基准从真实专利记录与专业材料整理，结合任务专属指标与专家评审标注，在四种 agent harness 下评测七个 LLM，结果显示现有系统在识别新颖性、评估风险和理解从业者意图上仍有困难。
正文
Abstract:Patent practice lies at the intersection of technical, legal, and linguistic work, making it challenging yet crucial to technological innovation. Recently, rapid advances in large language models (LLMs) and agentic systems have demonstrated the potential to complete realistic business workflows and even patent works. This raises a question: can current AI systems perform patent work reliably? Existing benchmarks cover a limited range of patent tasks and simplify evidence acquisition by providing reference materials or candidate documents. In this paper, we introduce PatentAgentBench, a benchmark comprising 510 instances across eight task suites: novelty search, freedom-to-operate (FTO) search, design FTO, invalidity search, claim charting, patent drafting, office-action response, and patent translation. Curated from real patent records and professional materials, the benchmark combines task-specific metrics with expert-reviewed annotations and rubrics. Retrieval tasks such novelty search, FTO, etc., require agents to search and compare patent collections without a supplied candidate pool. The drafting and OAR tasks assess the quality of agents' work given technical disclosure and office action notice. We evaluate seven LLMs across four agent harnesses on PatentAgentBench, finding that current systems still struggle to identify novelty, assess risk, and understand practitioner intent throughout patent workflows.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.35031 [cs.AI] |
| (or arXiv:2610.35031v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35031 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yijie Hu [view email]
[v1]
Mon, 26 Oct 2026 08:44:25 UTC (3,735 KB)
来源：arXiv cs.AI · arxiv.org