arXiv cs.AI· Bokai Pan, Ruijie Yang, Mingyue Cheng, Jiahao Wang, Xiaoyu Tao·· 1 天前AI 评分31
CastWorld：评估上下文感知智能体时间序列预测的基准
CastWorld: Benchmarking Context-Aware Agentic Time Series Forecasting Beyond Predictive Accuracy
AI 导读
CastWorld 是面向上下文感知时间序列预测的智能体基准，数据集覆盖 8 个领域、25 个真实数据集，并提供含 49 个核心工具与状态自适应循环的模块化 harness。实验显示，准确率相近的智能体决策行为差异明显，额外交互未必改善预测，智能体甚至可能选不出自己此前生成的最佳预测；在预测修正中允许获取证据可提升三个受测模型的任务感知预测分数。
正文 · 原文
Abstract:Agentic time series forecasting extends conventional approaches beyond one-shot numerical prediction toward autonomous, context-aware interaction. However, existing benchmarks offer limited integrated support for diverse contextual information, forecasting-specific adaptive interactions, and systematic evaluation of forecasting agents. In this paper, we introduce CastWorld, an agentic benchmark for context-aware time series forecasting. First, its context-rich dataset suite spans 8 domains and 25 real-world datasets, covering external variables, events, and historical cases under a unified context specification. Second, a lightweight, modular forecasting-native harness provides 49 core tools and a state-adaptive loop that tracks forecast versions and enables iterative evidence gathering and forecast revision. Intermediate forecasts can expose information gaps, while newly acquired evidence guides subsequent revisions and final version selection. Third, model-level evaluation assesses forecast quality, process execution, behavioral trustworthiness, and resource efficiency through decision trajectories and forecast-version histories. Also, systems-level evaluation compares predictive accuracy across forecasting systems. Experiments reveal distinct decision behaviors among agents with comparable accuracy. Additional interactions do not necessarily improve forecasts, and agents may fail to select their best previously generated predictions. Allowing evidence acquisition during forecast refinement improves task-aware forecast scores across all three evaluated models. CastWorld provides a reusable foundation for evaluating agentic time series forecasting under controlled conditions.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.15262 [cs.LG] |
| (or arXiv:2611.15262v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15262 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bokai Pan [view email]
[v1]
Wed, 11 Nov 2026 09:53:20 UTC (2,288 KB)
来源：arXiv cs.AI · arxiv.org