arXiv cs.CL· Haoyu Zhao, Sitian Tu, Yuxuan Huang, Huichi Zhou, Zhangyue Yin, Jiahao Huang, Jingyuan Xu, Tianmiao Shi, Jin Shi, Nianxi Zhang, Yangmiao Zhang, Zhuolang Chan, Huiyan Hu, Chenchao Sheng, Yaqing Luo, Yue Chen, Huaxiu Yao, Meng Fang, Jun Wang·· 14 小时前AI 评分45
Continual Learning Exam（CLE）：面向 LLM 智能体长程持续学习的基准测试
Continual Learning Exam: Benchmarking Long-Horizon RSI and Experiential Learning in LLM Agents
AI 导读
研究者推出 Continual Learning Exam（CLE）基准，由 50+ 领域专家构建 120 个人工撰写任务、每任务五级阶梯共 600 级，覆盖 12 个职业领域的 89 个岗位。
正文 · 原文
Authors:Haoyu Zhao, Sitian Tu, Yuxuan Huang, Huichi Zhou, Zhangyue Yin, Jiahao Huang, Jingyuan Xu, Tianmiao Shi, Jin Shi, Nianxi Zhang, Yangmiao Zhang, Zhuolang Chan, Huiyan Hu, Chenchao Sheng, Yaqing Luo, Yue Chen, Huaxiu Yao, Meng Fang, Jun Wang
Abstract:Continual learning is the ability of LLM agents to improve over time by acquiring, refining, and applying knowledge from experience while retaining useful prior knowledge. It is essential for agents learning over long time horizons, yet current evaluations largely assess individual task performance rather than improvement through accumulated experience. We introduce Continual Learning Exam (CLE), a large-scale benchmark developed by 50+ domain experts to measure experience-driven improvement in realistic professional workflows. CLE comprises 120 human-authored tasks, each organised as a five-level ladder, yielding 600 levels spanning 89 professional roles across 12 occupational domains. Agents must pass each level to unlock the next, with later levels building on experience accumulated during earlier assignments. An evaluation agent assesses each submission against human-authored rubrics. We compare a single-submission setting with one that provides diagnostic feedback after failure and allows one revision. Across 17 harness-model configurations using frontier LLMs, feedback and revision yield a median completion gain of 5.3 percentage points. Yet the best-performing system clears only 47.7% of levels and completes all five levels in just 23.3% of tasks, highlighting the difficulty of sustained learning from experience for existing frontier LLMs. As such, the dataset can also be used to post-train LLMs to strengthen their continual learning capabilities.
| Comments: | Project page: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.33956 [cs.LG] |
| (or arXiv:2610.33956v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33956 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haoyu Zhao [view email]
[v1]
Sun, 25 Oct 2026 00:00:25 UTC (12,887 KB)
来源：arXiv cs.CL · arxiv.org