Sam Altman 谈本地推理的浪费

Rohan Paul· @rohanpaul_ai · X·2026-10-30 07:24· 36 分钟前AI 评分47

AI 导读

Sam Altman 谈本地推理。 “从全球资源分配的角度看，让每个人、让世界上有限 DRAM 供应中的一大批都放在自己的电脑里、却只有 5% 的时间在用，这相当糟糕” 按每 token 算，数据中心轻松胜出。batch size 为 1 的本地模型按任何利用率指标看都是浪费，但它也是整个技术栈中唯一没人能对你弃用的部分。https://x.com/morqon/status/2115838856524120128/video/1

正文 · 原文

Sam Altman on local inference.

"from a global allocation perspective, it's quite terrible to have everybody, a bunch of the world's limited supply of DRAM sitting in their own computer and being used 5% of the time"

per token, the datacenter wins easily. A local model at batch size 1 is wasteful by any utilization metric, and it’s also the only part of the stack nobody can deprecate on you. https://x.com/morqon/status/2115838856524120128/video/1

来源：Rohan Paul · x.com