Meta 论文提出 Sharpening Tax：RL 后训练提升首试准确率但缩小解法覆盖
Meta 新论文发现，强化学习后训练让智能体首次尝试更准，但在最多 128 次尝试内找到解法的能力反而变差。研究在 3 个智能体基准上对比了 14 组基座与后训练模型，并构建了 PTGS 采样器，对难题提高温度、对简单题降低温度。
Rohan Paul· @rohanpaul_ai · X·2026-10-30 01:14· 50 分钟前AI 评分61
AI 导读
Meta 论文发现 RL 后训练让 Agent 首次尝试更准，但在最多 128 次尝试内找到解法的能力反而变差。
正文 · 原文
New Meta paper finds Reinforcement learning post-training makes an agent better on its first attempt and worse at ever finding a solution when it gets 128 attempts.
People assume reinforcement learning teaches agents new skills like tool calling.
That claim comes mostly from math and coding, where base models have already seen huge amounts of data.
They ran 14 base and post-trained model pairs on 3 agentic benchmarks with up to 128 tries per task. They also built PTGS, a sampler that raises temperature on hard prompts and lowers it on easy ones.
On WebShop a gemma-4-31B base model solved over 85% of tasks within 128 tries against 56% for the post-trained version. Their Sharpening Tax came out positive in 36 of 42 model and benchmark pairs, meaning coverage shrank.
Before picking a post-trained model for an agent that can retry, measure how many tasks it ever solves, not just first-try accuracy.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news