UC Berkeley 等机构论文：智能体自改 harness 提分的同时会丢掉原本能完成的任务

Rohan Paul· @rohanpaul_ai · X·2026-11-09 18:56· 48 分钟前AI 评分54

AI 导读

UC Berkeley、中国科学技术大学等机构的论文发现，通过 harness evolution 在模型冻结情况下自改提示词、技能、工具或代码的智能体，平均分提高的同时会丢失原本能完成的任务。

正文 · 原文

New University of California, Berkeley+other top labs paper finds that agents that rewrite their own harness to score higher also lose tasks they once solved, which the average score hides.

Harness evolution lets an agent edit its own prompts, skills, tools, or code while the model stays frozen. Results usually come as 1 higher benchmark score, which can hide lost tasks, higher costs, and gains that don't carry over.

They ran 4 evolution methods, including GEPA and HyperAgents, with GPT-5.4 on 2 harnesses and 2 benchmarks. Then they re-tested each evolved agent 8 times against its starting version.

All 16 evolved agents gained some tasks and lost others, and 1 raised reliability by 5.2 points while losing 10 tasks it used to cover. Of 11 agents that became more reliable, 8 also cost more per run.

When you evolve an agent harness, track tasks lost, cost per run, and results on unseen tasks, not just the average score.

来源：Rohan Paul · x.com