跳到正文
arXiv cs.AI· Zheng Fang, Yiqun Sun, Simin Qu, Shijie Jiang·· 10 小时前AI 评分55

HydroMind:可核验的水文研究自主智能体

HydroMind: A Checkable Agent for Autonomous Hydrological Research

AI 导读

研究团队提出水文建模系统 HydroMind,并对比三种 harness 下同一智能体自主研究四个水文建模开放问题的表现。三种设置分别提交 23、20、41 条模型优劣结论,经实验自身代码与预测复算后不成立的占比为 30%、20%、5%,提供统计检验或置信区间等可核验依据的占比为 48%、53%、83%。在语言模型不变的前提下,仅改变工具与流程约束即可减少不成立的结论并提升可核验比例。

正文 · 原文

View PDF

Abstract:General-purpose coding agents have written code for hydrological and Earth-system models. In these studies the researchers decided in advance how the model would be evaluated, for example the test period, the scoring function, and the standard the model had to meet. No study has tested what happens when the agent makes these decisions itself: how many of its claims are supported by its own data, code, and results, and whether readers can check them. Both may depend on the agent's harness, that is, the tools it can call and the written instructions it follows. We compared three harnesses: the agent's own harness; this harness plus the modeling tools of HydroMind, a hydrological modeling system we developed for general-purpose agents, which run every model through fixed commands and store its settings, trained model, and predictions; and both plus HydroMind's research procedure, under which the agent writes a research design before running any model and records every change to it. Under each harness, the same agent studied four open questions in hydrological model development and was given only the data. Under the three harnesses, the experiments submitted 23, 20, and 41 claims, each stating that one model or approach is better than, worse than, or similar to another. A claim does not hold if the difference recomputed from the experiment's own code and predictions disagrees with it, or if the code did not compare the two models or approaches the claim names. Under the three harnesses, 30%, 20%, and 5% of the claims did not hold. A statistical test, confidence interval, or decision rule that lets readers judge the stated difference was reported for 48%, 53%, and 83% of the claims. With the language model unchanged, changing only the harness reduced the share of claims that do not hold and increased the share that readers can check.
Comments: 16 pages, 4 figures, 1 table
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Geophysics (physics.geo-ph)
Cite as: arXiv:2611.23323 [cs.SE]
  (or arXiv:2611.23323v1 [cs.SE] for this version)
  https://doi.org/10.48550/arXiv.2611.23323

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zheng Fang [view email]
[v1] Sun, 15 Nov 2026 17:53:30 UTC (778 KB)

来源:arXiv cs.AI · arxiv.org