arXiv cs.CL· Yue Zhao, Xiyang Hu, Zuobin Xiong, Zhangyu Wang, Ruolin Li·· 9 天前AI 评分24
AI4Fire：大语言模型在野火任务上的评测
AI4Fire: Evaluating Large Language Models on Wildfire Tasks
AI 导读
AI4Fire 对 6 个核心模型在 5 项野火任务上做零样本评测，并对比"裸跑"与"有依据"两种设置，另扩展 29 个模型。加入只读 SQL 工具后，所有核心模型的数据库准确率从最高 16% 提升到至少 88%；但无核心模型能胜过直接重复当日人力配置这一简单规则，两个开放权重模型大多照抄历史火灾日的中位数。
正文 · 原文
Abstract:Large language models (LLMs) are entering wildfire management, where overstated evaluations can cost property and lives. How do they perform on wildfire tasks, with and without grounding? Bare means a model receives the task input alone. Grounded means it also receives one task-specific addition: for smoke detection, a smoke-free reference frame from the same camera. AI4Fire runs six core models bare and grounded on five wildfire tasks, zero-shot; a sweep adds 29 more. Our literature search on fire tasks found 138 works; none combines this roster, task coverage, and paired bare and grounded runs. We report three findings. (1) Grounding helped most where the addition carried the answer: a read-only SQL tool lifted every core model's database accuracy from at most 16 to at least 88 percent. (2) Simple rules were hard to beat: no core model outperformed repeating today's staffing count, and two open-weight models mostly copied the median of similar earlier fire-days, a worse forecast. (3) Public releases carry hazards: a fire-danger column separates the holdout perfectly, and 67 aerial fire frames carry smoldering or fire-free labels read from a clipped thermal maximum. We release prompts, responses, scores, code, and the survey record.
| Comments: | 51 pages. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.10946 [cs.CL] |
| (or arXiv:2610.10946v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10946 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ruolin Li [view email]
[v1]
Wed, 7 Oct 2026 21:52:18 UTC (1,322 KB)
来源：arXiv cs.CL · arxiv.org