arXiv cs.AI· Leo Y. Lin, Vishakha Ramani, Z. Berkay Celik, Paul Castro, Marquita Ellis·· 3 天前AI 评分25
ROAR：统一异构 AI 驱动研究系统的运行数据
ROAR: Unifying Runs across Heterogeneous AI-Driven Research Systems
AI 导读
研究团队提出 ROAR，用于系统化统一并分析异构 AI 驱动研究系统（ADRS）的输出。它通过关系型 schema 与解析层归一化不同格式的结果，同时保留数据血缘与时间结构，并支持无需改 schema 接入新系统。基于多个 ADRS 的 900 多次运行语料，ROAR 揭示了相同配置运行可能收敛到不同分数、多数运行早期即获得大部分收益等难以从单一系统观察到的规律。
正文 · 原文
Abstract:Each run of an AI-driven research system (ADRS) is an expensive search over a vast solution space, and dependable evaluation requires many runs, making run data both costly to produce and valuable to retain for large-scale analysis. Yet this data remains fragmented: teams operate in isolation, ADRS frameworks emit results in different formats, and no shared infrastructure exists to aggregate or compare runs across problems and systems. We present ROAR, a solution for systematically unifying and analyzing heterogeneous ADRS outputs. ROAR addresses two challenges: reconciling heterogeneous ADRS outputs and enabling analytics across runs with different objectives and scoring functions. We achieve this through a relational schema and parsing layer that normalize heterogeneous ADRS outputs while preserving data lineage and temporal structure, and accommodating new systems without requiring schema modifications. From building a corpus of more than 900 runs from multiple ADRS, we show how pooled data can reveal properties of problem landscapes that are difficult to observe. Consistent with prior work, runs with identical configurations may converge to different scores. We find that many runs realize most gains early, and that the effectiveness of different strategies for incorporating prior solutions into the search process varies across problems. We further show that the pooled corpus is actionable and not merely analytical by using ROAR to configure ADRS runs. Together, these results illustrate how pooled ADRS data can expose problem-dependent structure in search behavior that is difficult to detect from any single system, team, or benchmark. Such cross-cutting insights are difficult to obtain while runs remain siloed; ROAR is the first infrastructure designed to unify them.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.03966 [cs.AI] |
| (or arXiv:2610.03966v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03966 arXiv-issued DOI via DataCite |
Submission history
From: Leo Lin [view email]
[v1]
Fri, 2 Oct 2026 19:21:07 UTC (6,924 KB)
[v2]
Tue, 13 Oct 2026 14:44:07 UTC (6,924 KB)
来源：arXiv cs.AI · arxiv.org