arXiv cs.CY· Isaac Corley, Nils Lehmann, Caleb Robinson, Gabriel Tseng, Anthony Fuller, Hamed Alemohammad, Evan Shelhamer, Jennifer Marcus, Hannah Kerner·· 11 小时前AI 评分34
没人知道地理空间基础模型（GFM）的 SOTA 是什么：152 篇论文审计揭示评测乱象
No One Knows the State of the Art in Geospatial Foundation Models
AI 导读
一项对 152 篇地理空间基础模型（GFM）论文的审计发现，该领域无人能确定当前 SOTA：论文间存在 47 处至少 5 个百分点的结论分歧（其中 18 处达 10 个百分点以上），126 篇使用预训练数据的论文中有 94 篇的配置独一无二，39% 的论文未发布模型权重。
正文 · 原文
Abstract:Geospatial foundation models (GFMs) have been proposed as generalizable backbones for disaster response, land-cover mapping, food-security monitoring, and other high-stakes Earth-observation tasks. Yet the published work about these models does not give reviewers or users enough information to tell which model fits a given task. We argue that nobody knows what the current state of the art is in geospatial foundation models. The methods may be useful, but the GFM literature does not standardize evaluations, training and testing protocols, released weights, or pretraining controls well enough for anyone to compare or rank them. In a 152-paper audit, we find 47 cross-paper disagreements of at least 5 percentage points, including 18 of at least 10 points, within model, benchmark, and protocol groups; 94/126 papers with pretraining data use a configuration no other paper uses; and 39% of GFM papers release no model weights. This lack of community standards can be solved. We propose six concrete expectations: named-license weight release, shared core evaluations, copied-versus-rerun baseline annotations, variance reporting, one shared evaluation harness, and data-vs-architecture-vs-algorithm controls. These gaps are a coordination failure, not a fault of any individual lab; the authors of this paper, like many others in the GFM community, have contributed to them. Rather than just critiquing the community, we aim to provide concrete steps toward a shared understanding of how to evaluate and improve GFMs.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computers and Society (cs.CY) |
| Cite as: | arXiv:2605.12678 [cs.CV] |
| (or arXiv:2605.12678v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12678 arXiv-issued DOI via DataCite |
Submission history
From: Isaac Corley [view email]
[v1]
Tue, 12 May 2026 19:29:51 UTC (69 KB)
[v2]
Fri, 12 Jun 2026 18:15:18 UTC (69 KB)
[v3]
Thu, 10 Sep 2026 19:44:05 UTC (79 KB)
[v4]
Fri, 6 Nov 2026 02:25:43 UTC (88 KB)
来源：arXiv cs.CY · arxiv.org