MIT 的 Alex Zhang 谈 RLM、GPU 内核与智能体集群研究

Last call for regular tickets for AI Engineer NYC! As an exclusive for Latent Space subscribers, the first 30 of you can take a 30% off code if it helps - for new tickets only, no refunds! See you in 2 weeks!

虽然我们倾向于在播客中报道行业动态，但偶尔也会庆祝一位明显正在崛起的明星博士生。在 2024 年，我们介绍了 Shunyu Yao，他后来在 OpenAI 打造了 Operator，现在是腾讯首席 AI 科学家。在 2025 年，我们介绍了 Jack Morris，他后来联合创立了 Engram，估值 6 亿美元，如今是持续学习领域的领军人物。

今年我们很荣幸介绍 MIT 的 Alex Zhang 的工作。

从 GPU kernel 和 KernelBench 到 递归语言模型、被管理不善的天才们，以及大规模多智能体集群，Alex Zhang 正在探索：把越来越强大的模型包裹在原始系统中，我们究竟在桌面上留下了多少能力。

RLM 在今年年初刷屏了时间线：

而一个基于 RLM 的 harness 是第一个在 OpenAI 的 Astra 之前约等于解决 ARC-AGI-3 的：

甚至到今天，它还在影响比 RLM 具有更极端影响的新研究：

我们深入探讨 GPU Mode 和 AI 编写的 kernel、研究品味以及为什么学者应该下工业实验室不会下的赌注、GEV 和标准自回归语言模型的替代方案，以及将 harness 视为组合泛化器的理念。Alex 解释了 RLM、上下文卸载、程序化子智能体调用、Prime Agent、持久子智能体，以及为什么未来的“语言模型”实际上可能是一个隐藏在简单界面之下的隐形智能体集群。我们还讨论了 OpenAI 的大规模智能体实验、Kimi 集群、Sakana AI 的开放式研究、推测性程序化工具调用、能力悬置、Neuralese，以及 Alex 认为下一个重大研究机会可能在哪里。

我们讨论：

• 为什么AI 生成的 GPU kernel仍然为人类专业知识留下大量空间

• 一个专家洞见如何可能取代大量暴力 token 搜索

• 为什么博士生应该下那些最初看起来微不足道、奇怪或无意义的研究赌注

• SWE-bench、RLM、ReAct 和 Quiet-STaR揭示了关于研究品味的什么

• GEV，以及为什么语言模型不一定意味着自回归的文本到文本解码器

• 为什么Claude Code、Codex 和 Pi在结构上比它们看起来更相似

• harness 设计如何改善跨任务和领域的组合泛化

• RLM：上下文卸载、代码执行、递归子智能体和共享内存

• Prime Agent、持续 harness，以及持久的智能体间通信

• 为什么你未来查询的模型可能秘密地是一整个集群或脚手架

• OpenAI 的10,000 智能体实验、130B 输出 token 和约 4000 万美元等值的问题求解

• 为什么智能体集群的很大一部分可能是浪费的搜索——以及为什么收敛仍然很难

• Kimi 与 OpenAI以及多智能体系统的不同方法

• 开放性、Sakana AI，以及在大量生成工作中寻找隐藏的珍宝

• 为什么当前的前沿模型可能已经拥有巨大的能力过剩

• 推测性程序化工具调用以及与生成重叠的工具执行

• 英语、代码，还是一种全新的“Neuralese”是否制约了模型的推理方式

• AI for science、快速迭代的基准测试，以及 Alex 如何选择押注哪些研究问题

Alex Zhang

• 网站：alexzhang13.github.io

• X：@a1zhang

时间戳

00:00:00 介绍

00:00:49 GPU Mode、KernelBench 与 AI 编写的内核

00:07:38 人类专业知识 vs. 暴力 AI 搜索

00:13:20 研究品味与大胆押注

00:19:28 GEV 与重新思考语言模型

00:29:03 电子游戏智能体与 harness 问题

00:31:01 为什么 Claude Code、Codex 和 Pi 如此相似

00:36:42 作为组合泛化器的 harness

00:44:24 RLM 详解

00:52:01 Prime Agent 与持久化子智能体

00:57:41 RLM 的实际应用

01:00:30 OpenAI Swarms 与语言模型的未来

01:07:26 开放式与 Sakana AI

01:15:52 Kimi vs. OpenAI Agent Swarms

01:20:06 能力过剩与推测性工具调用

01:28:19 Neuralese、未来研究与 AI for Science

文字记录

介绍：Alex Zhang、RLM 与 GPU Mode

Swyx [00:00:00]： 好的，我们和 Alex Zhang 一起在演播室，我想他最出名的是 RLM，但你还有几个其他身份。欢迎来到节目。

Alex Zhang [00:00:12]： 是的。谢谢你邀请我。

Swyx [00:00:13]： 是的。我想还有 GPU Mode？

Alex Zhang [00:00:15]： 是的，还有 GPU Mode。

Swyx [00:00:16]： 你是被 Mark Saroufim 引荐进来的。不是每个人都能得到这种欢迎。

Alex Zhang [00:00:19]： 对。是的。是的。我和 GPU Mode 里的所有人都很熟，所以是的

Swyx [00:00:23]： 是的

Alex Zhang [00:00:23]： 我们经常以各种身份一起合作，甚至不只是 GPU Mode 本身，所以。

Swyx [00:00:29]： 是的。我们能解释一下吗，这样那些不太了解的人就不知道这件事。它只是一个——它是一个 Discord。它过去专注于，我想是 CUDA Mode，然后稍微泛化了一些。它是由 Mark 发起的。

Alex Zhang [00:00:41]： 对。

Swyx [00:00:42]： 它基本上就像。对我来说，它就像 PyTorch 团队的招聘渠道。

Swyx [00:00:45]： 然后你离开了 PyTorch。

Alex Zhang [00:00:47]： 对。

Alex Zhang [00:00:49]： 是的。所以它过去是，我想它实际上是在我上大学的时候开始的，大概在 2023 年。我想它是由 Mark、Andreas 和 Jeremy Howard 发起的。最初的设想就是，它是一个 GPU——或者说它是一个专门用于学习如何编写 GPU 内核的 Discord，他们还有讲座。基本上就是这些。我之所以对它感兴趣，是因为我当时在写 GPU 内核。实际上那完全是出于偶然。我当时在 Snapchat 实习，然后

从 CUDA Mode 到 GPU Mode

Swyx [00:01:22]： Rexis。

Alex Zhang [00:01:23]: 是的，我对 Rexis 感到非常厌倦。所以，他们有一个项目，比如，他们对写作感兴趣。就是这篇叫 Infinite Attention 的论文。它像是谷歌的一篇论文。

Swyx [00:01:35]: 是的，我们在 Paper Club 上讨论过它。

Alex Zhang [00:01:36]: 是的，对。所以我很感兴趣，你是否能在 Snapchat 为它编写专门的 kernel。结果并没有。没什么真正成果，但我加入了 GPU Mode。当时它叫 CUDA Mode，我想是出于法律原因之类的，他们改了名字。但我遇到了 Mark，遇到了 Matei，遇到了很多其他深度参与这个社区的人。然后 Mark 提出了一个叫 Popcorn 的想法，就是你们今天看到的排行榜。但总体思路是，我觉得我们所有人都有这种直觉，GPU 编程非常像，如果你们做过竞技编程的话。这不是，我不是说它们是可迁移的技能。

Popcorn、KernelBench 与自动化 GPU Kernel

Swyx [00:02:17]: 你有约束条件。你稍微做点代码高尔夫。

Alex Zhang [00:02:19]: 是的。

Swyx [00:02:19]: 是的。

Alex Zhang [00:02:19]: 是的。而且有，实际上人们做的优化空间小得惊人。而且实际上人们有兴趣优化的 kernel 本身并没有那么多。所以我们有了这个想法，如果你有足够的数据，就像 Codeforces 有数百万道题一样。如果你能用 GPU 代码做到这一点，你就能扩展并自动化 GPU kernel 开发，这对研究人员来说是件大事。因为我认为最大的瓶颈之一，比如，如果你看 Mamba，他们发布论文时附带 kernel，因为否则你没法以任何有意义的方式使用它？而且不是每个人团队里都有一个 Tri Dao。所以我们对此非常感兴趣。KernelBench 也是由此诞生的，就是，我们能不能让 LLM 自动化 GPU kernel 代码？我觉得那是一段非常有趣的时光。那是在大学和我的博士之间，是的，我在 GPU Mode 做事非常愉快。现在我只是偶尔帮忙做讲座。我没那么参与了，而且我觉得总的来说，我们不再像以前那样有那么多比赛了。但是，是的，我仍然和那里的每个人保持很多联系。

Swyx [00:03:27]: 有友好的竞争吗？因为我认为以前做这个的社区像是 MLSys、MLPerf

Alex Zhang [00:03:32]: 是的

Swyx [00:03:32]: 之类的。有友好的竞争吗？这就像是新一代的 MLPerf，还是怎么回事？

Alex Zhang [00:03:38]: GPU Mode 的好处在于它也是一个社区，很多讲座对初学者来说都很容易跟上、提问之类的。而且那些竞赛，怎么说呢，不是次要的，但你可以参与其中来学习。我觉得像很多 MLSys、MLPerf 这类基准测试，大部分情况下只有严肃的实验室和公司才会参与。至少是认真参与。这是我对此的理解。我可能说错了。但我觉得现在除了 GPU Mode 之外，真正令人兴奋的一点是，有更多的网站，还有更多人在做竞赛主办这件事。我觉得有这么一个。

Alex Zhang [00:04:21]: 我记得有个网站叫 LeetGPU 之类的，就像是 GPU 问题的 leetcode。

Swyx [00:04:26]: 哇。

Alex Zhang [00:04:26]: 还有其他的，我见过。我们见过。有很多类似的东西涌现出来，还在 GPU Mode 上聊过，这非常令人兴奋，因为我觉得 GPU 编程以前超级小众，至少在我对它感兴趣的时候是这样。我之所以对它感兴趣，唯一的原因是 Tri Dao 在普林斯顿做了一场讲座，因为他当时在申请教职。他现在是那里的教职了，但我在 2023 年左右听了他的 FlashAttention 讲座，我当时就想，“哇，这是有史以来最酷的东西。”

Alex Zhang [00:04:56]: 我当时想，“这就是。这就是所有人都应该去做的事。”我猜 vLLM 之类的东西也出来了，然后就想，“哦，我们应该写 kernel。”但现在呢，每个人都在写 kernel。每个人。这，这。我觉得作为一个领域，它在某种意义上几乎已经饱和了。

AI 编写的 Kernel 与验证鸿沟

Vibhu [00:05:11]: 对于想入门的人，有什么有趣的观点吗？我觉得最大的新闻之一是 GPT-5.6 写出了更高效的 kernel，所以 Terra 和 Luna 可能便宜 80%。

Alex Zhang [00:05:24]: 是的。

Vibhu [00:05:25]: 然后我们还看到其他竞赛，人们在创纪录，他们说，“我们在跑某种自动研究循环，”而这些人并没有

Alex Zhang [00:05:34]: 对

Vibhu [00:05:34]: 任何写 kernel 的背景，对吧？

Alex Zhang [00:05:36]: 是的。所以即使在 GPU Mode 排行榜上，如果你看看最近的很多问题，几乎所有的解决方案都是 AI 生成的。不过，你会注意到在排行榜上。有个叫 Gauners 的人，他是 GPU Mode 的常客。我们早就知道他是个超级厉害的 GPU 内核编写者。我们在这个排行榜上发现的一件事是，几乎——他也用 AI 来帮助他完成这些解决方案，但大部分情况下，是他来提示并引导它往某些方向走。我们发现，他的内核基本上是前 10 名中唯一在实际的端到端系统中真正稳定的。这确实让人质疑，这不是……就像 GPU 内核存在验证问题。我们多少知道这一点。自 KernelBench 发布以来，这就一直是个问题。有很多奖励黑客行为。但你也——是的，你也会注意到，代码行数少了很多，但

Vibhu [00:06:33]: 是的，我正想问，这明显吗，还是只是

Alex Zhang [00:06:36]: 是的，不，这，这

Vibhu [00:06:36]: 好吧

Alex Zhang [00:06:36]: 这绝对非常重要，而且我觉得，有趣的是，擅长编写 GPU 内核仍然有很大的优势空间。

Vibhu [00:06:44]: 好吧，所以从验证到

Alex Zhang [00:06:45]: 确实有，是的。我觉得，而且这也适用于很多 AI 系统。我觉得，即使是最新的数学证明之类的，也不一定意味着数学家就过时了。这些公司仍然雇佣数学家，去做，无论是数据标注工作，还是仅仅引导模型解决问题。在这些事情上有知识仍然有很大的优势空间，所以。

Swyx [00:07:12]: 只是知识吗，还是说也有更多的规划，是否存在一种更有效的、涌现出来的规划风格？

Alex Zhang [00:07:22]: 我觉得这是混合的。也许这就是你的意思，比如对

Swyx [00:07:27]: 差不多是这样

Alex Zhang [00:07:28]: 如何解决问题的直觉。

Swyx [00:07:29]: 比如，我总是把我的代码画成图。

Alex Zhang [00:07:31]: 是的。

Swyx [00:07:31]: 对吧？

Alex Zhang [00:07:31]: 是的。

Swyx [00:07:31]: 然后，如果图里有我不理解的部分，我就一直研究直到理解为止。否则，就不允许。

Alex Zhang [00:07:37]: 是的。

Swyx [00:07:38]: 是的。

Alex Zhang [00:07:38]: 所以我认为，这就像是那些东西的混合体，比如，那些工作的人。就像，那些知道如何看待这些问题以及如何解决它们的人，也懂得如何使用 AI 来完成这些任务。因为，就像，你扮演着一个非常强大的验证者。就像，如果你知识渊博——或者如果你知道该做什么，而且你也是。就像，我认为我们通过这些智能体集群之类的东西发现的是，当你为一个问题投入足够的计算资源时，你就能充分探索该问题的解决方案。但很多时候，也许你可以在某件事上烧掉 1000 亿或 1 万亿个 token，但如果你引入一个对这个问题有所了解的人，他们就能为模型揭示一些东西，从而抹去那 1 万亿 token 的消耗。这里的情况并不是特别清楚，比如，趋势到底是什么。但我认为，就像，现在野外还有太多我们想要解决的问题，而且我们无法承受总是尽可能多地投入计算资源。就像，所有这些东西中仍然有一个效率方面，这非常重要。

光速极限、内存与巨型内核

Swyx [00:08:41]: 是否存在一个理论上的正确答案，你只需基于物理计算，然后就能接近物理极限？

Alex Zhang [00:08:49]: 是的。所以对于 GPU 内核，你可以计算。实际上，有时计算起来并不那么容易，比如，取决于问题有多复杂。就像，对于矩阵乘法，计算起来非常容易，这种类似光速的估计，即最快内核能有多快。而且，这也假设，也许你所有的数据都从 CPU 开始，或者也许从 GPU 上的 DRAM 开始，等等。就像，这会稍微改变这些数字，但

Swyx [00:09:18]: 传输和所有这些事情，是的。

Alex Zhang [00:09:20]: 是的。但我要说，不过，在很多情况下，是否有可能达到这个理论数字并不清楚，如果这说得通的话。就像，这假设了数据的完美重叠和传输，而且，也许存在一些你无法绕过的瓶颈。但通常，内核甚至都差得远。就像，我们写的那些离这个数字还差得远，根本谈不上有意义。

Swyx [00:09:43]: 是的。那么，重要的是速度吗？你是否也关心，显然，内存，这

Alex Zhang [00:09:48]: 嗯

Swyx [00:09:48]: 会影响速度？你关心功耗吗？所以，我今年最喜欢的播客之一，是 Geoff Dean，他说：“实际上，我只是追踪微焦耳，或者，纳焦耳、皮焦耳。”

Alex Zhang [00:09:59]: 是的，如今通常是皮焦耳。

Swyx [00:10:01]: 皮焦耳。

Alex Zhang [00:10:01]: 是的。

Swyx [00:10:01]: 你关心这个吗？

Alex Zhang [00:10:03]: 所以我不关心。

Alex Zhang [00:10:04]: 是的。我想也许我并不是，我并不是那么

Swyx [00:10:06]: 但这里的一切都是速度，对吧？就像

Alex Zhang [00:10:07]: 这里的一切都是速度

Swyx [00:10:08]: 没人在数皮焦耳。

Alex Zhang [00:10:09]: 但——这里有个需要注意的地方，我认为，单个 kernel 的语境下存在速度，而更大问题的语境下也存在速度，比如 N10 模型。因为，需要考虑的一点，这也是为什么讨论光速指的是什么很重要，因为在这些 kernel 的情况下，我们总是从所有东西都在 HBM 里开始，对吧？但你可以想象，一个端到端的模型，你可能想在两层之间做的事情是，牺牲第一个操作的速度，把东西保留在缓存里供第二个操作使用。而这些事情，你无法真正孤立地通过这些 kernel 来实现。人们把这叫做融合，或者

Swyx [00:11:01]: 巨型 kernel

Alex Zhang [00:11:02]: Kernel 融合问题。对，或者类似巨型 kernel 的东西。它通常只在大多数情况下你受限于内存时才适用。但这也是一个问题，随着这些模型变得更好，我们是否应该直接生成巨型 kernel？那是我们想要的吗？

Vibhu [00:11:20]: 你怎么看？

Alex Zhang [00:11:21]: 我认为这真的很难，因为你需要数据来做这件事。而且我觉得，我还没有在实际中见过这样的例子：在没有任何示例的情况下，我们引导出解决一类非常困难问题的能力。我认为这可能不那么有趣的另一个原因是，在单个 kernel 的层面上，A，它们没那么复杂，但 B，你多少能确信单个 kernel 里没有那么多结构。但是，在巨型 kernel 里，我更倾向于相信编译器在这里会更好。某种基于更高层操作的编译器是合理的，因为总的来说，实际上，我认为巨型 kernel 非常，各个部分非常可组合，就像单个 kernel 一样。有些领域你可能想做奇怪的融合之类的，但总的来说，我认为这些情况编译器可能能处理。据我所知，有一家公司正在做这个，他们也在 GPU mode 上做过一些演讲。

Swyx [00:12:28]: 对，我想基本上把所有 GPU mode 的讨论都集中在这里，因为显然还有其他部分

Alex Zhang [00:12:32]: 对。

Swyx [00:12:32]: 我们需要继续。

Vibhu [00:12:33]: 我觉得有东西可以提一下。你们确实举办了很多非常好的讲座。都在 YouTube 上。人们可以关注。而且你

Alex Zhang [00:12:39]: 是的

Vibhu [00:12:39]: 主导了相当一部分。你仍然参与很多。

Alex Zhang [00:12:41]: 我以前是。有时我仍然会。我觉得主要是 Mark。Mark 通常是做这些的人。Matei 有时也会，但是，是的，我强烈推荐它们。它们是非常好的资源。我觉得人们在那里分享的东西多得有点疯狂，所以是的。

Swyx [00:12:59]: 因为如果你在那里，你就是非常，你就是完全正确的受众？

Alex Zhang [00:13:03]: 是的，完全正确。

Swyx [00:13:03]： 就像这不会成为主流。

Alex Zhang [00:13:04]： 而且还有很多入门材料，我们也放上去了，我觉得对人们很有用。

基准测试、普林斯顿与研究品味

Swyx [00:13:10]： KernelBench 挺有影响力的。我就想看看，那是去年的事了。

Alex Zhang [00:13:13]： 是的。

Swyx [00:13:14]： 还有什么正在进行的工作你想提一下，让大家关注？因为显然你参与了这个

Alex Zhang [00:13:20]： 是的

Swyx [00:13:20]： 领域。

Alex Zhang [00:13:20]： 对。我或许可以讲讲背景故事，其实我参与了很多基准测试，或者说以前参与过，可能是在我读博之前。这始于我在普林斯顿的时候。我和那里的 SWE-bench 团队合作。

Swyx [00:13:33]： John、Carlos

Alex Zhang [00:13:33]： John、Carlos

Swyx [00:13:34]： Ofir

Alex Zhang [00:13:34]： 还有 Ofir。他们都很棒。就像

Swyx [00:13:36]： Karthik

Alex Zhang [00:13:36]： 我很喜欢他们。是的。

Swyx [00:13:37]： 基本上就是这样，我觉得人们不了解有多少基准测试来自同一个团队

Alex Zhang [00:13:42]： 是的

Swyx [00:13:42]： 在普林斯顿。

Alex Zhang [00:13:44]： 这太疯狂了。

Swyx [00:13:45]： 是 Xun Yu 吗？

Alex Zhang [00:13:45]： 是的。对。

Swyx [00:13:46]： 我们之前请他上过播客。现在他好像在管理腾讯。

Alex Zhang [00:13:48]： 是的，现在他就像，他就像个超级明星。我遇到他的时候，他在指导我的朋友 Michael Tang，他现在在 Anthropic，但他们合作很多。我们当时是 Karthik 实验室里的两个本科生。我——后来还有其他人加入。但没错，Xun Yu 很棒。我直到后来，离开之后才知道他这么厉害，但

Swyx [00:14:12]： 是的。就像，好吧，博士生很少。像你这样的就是下一个。就像每年，我们会介绍一个人，基本上整个博士期间都目标明确。

Swyx [00:14:25]： 这样的人不多。Xun Yu 显然是其中之一。还有 Jack Morris 是另一个。而且，我们在节目之前聊过研究品味。

Swyx [00:14:33]： 对吧？就像不知怎么的，有些研究生就是有着非常幸运的职业生涯，就像，是的，大多就像，对，这会是持久的、相关的，大家都应该知道。

Alex Zhang [00:14:42]： 是的。

Swyx [00:14:42]： 而其他人，就什么都没有。

Alex Zhang [00:14:44]： 我觉得即使在产业实验室里的人也是如此。我觉得只是研究生更显眼。所以你就看到，就像你看到，有些人

Swyx [00:14:56]： 是的，你可以发表

Alex Zhang [00:14:57]: 真正喜欢碰运气的人，或者说，这是运气和非常聪明的混合，诸如此类。我觉得研究品味也是如此，至少在我看来，它是通过机会发展起来的。比如我——我非常幸运地走上了我走的这条路，比如在普林斯顿工作，然后后来，比如在 MIT 找到了我的 Omar。他是一位出色的导师。不过我要说，我发现研究生或学术界最成功的研究，往往出现在人们关心那些工业界大多数人可能没有关注的问题时。我认为这就是问题所在，很多研究生做的东西是为了受益，比如在工业实验室看起来不错。例如，他们会研究一些现在非常流行的基准测试。我觉得 2023 年的基准测试和现在的基准测试是完全不同的情况。有很多人研究像 harnesses、meta-harnesses 以及针对 XYZ 任务的特定 harnesses。当你真正思考时，一个人研究这个的原因可能是，围绕我们今天拥有的模型有一个明确的目标，比如这就是我想看到的。但比如，我会举 RLM，即递归语言模型论文为例，因为我觉得这是一个超级简单的想法。我觉得当它刚出来时，也有很多人，当他们看到这样的东西时，他们会说：“这到底有什么目的？”

Swyx [00:16:27]: 或者太酷了。

Alex Zhang [00:16:28]: 是的。比如为什么，比如什么？这只是子代理之类的，对吧？

Alex Zhang [00:16:32]: 我觉得当你得到那样的反应时，这几乎是一个好迹象，因为很明显人们没有在思考这件事的目的是什么。我再举一个 SWE-bench 的例子。当 SWE-bench 刚出来时，Ofir 很喜欢讲这个故事。它刚出来时，没人在意。大家都说，“这是一个不可能完成的任务。我们为什么要把它当作一个基准来考虑？”直到 Devin 出现，大家才说，“哇，这是我们可以去攀登的东西。”我觉得这很真实。你往往会看到很多想法。我想，我最喜欢的例子是 Eric Seligman 的工作，比如 STaR 和 Quiet-STaR。我觉得当你读那篇论文时，至少我第一次读的时候，我想，“这难道不是一个显而易见的想法吗？”也许不是。我不知道。我当时想，“哦，这看起来真的很简单。”或者像思维链，也是一样。或者像 Xun Yu 的 react。就像，好吧，是的，当然。但当你真正思考时，就会问为什么。这篇论文的价值是什么？我认为它来自于，它讲述了一个关于你希望这个领域变成什么样的故事。而在学术界，这非常难做到，因为如果你看所有这些论文，Quiet-STaR、ReAct、RLMs、SWE-bench，这些论文都不是。它不像 GPT-6 Astro 发布？不是每个人都像，“哦天哪，我现在要用这个，这是世界上最好的东西。”学术界就是负担不起这样做，至少现在不行。我。我认为这应该改变，有整整一堆理由，但我觉得就像。如果你没有。作为一名博士生，我认为你处于一个非常独特的位置，你可以基本上研究任何你想研究的东西。如果你不利用这一点，去研究那些没人关心或者人们视为琐碎的东西，比如，“哦，我想过这个，但我不使用它，”我只是觉得，最终，这项研究永远不会那么有趣，因为如果你想留在学术界，你需要下大赌注。因为否则，我觉得，就去工业实验室吧。他们有大量的资源，大量的人才。为什么要在一个你没有很多资源、甚至周围都没多少人的领域里限制自己？我认为这真的就归结为，大赌注。就像，你必须下大赌注，而且很多会失败？就像，那是自然的。但我认为

Alex Zhang [00:18:55]: 那就是，作为一名博士生，这是你相对于其他实验室任何一个人最大的优势，因为你不必处理官僚主义和其他所有这些事情。

Swyx [00:19:07]: 有道理。

Alex Zhang [00:19:07]: 是的。

Swyx [00:19:08]: 我问过很多人这个问题，通常他们都会敷衍过去。所以我觉得。我很感激你实际上给出了一个深思熟虑的回答，就像，不，这是你的不公平优势，因为基本上其他一切对你都不利。

Jev 与打破自回归解码器范式

Alex Zhang [00:19:20]: 是的，没错。所以，老实说。我会举 Jev 作为例子，因为它不是学术性的

Swyx [00:19:26]: 哇，好吧。

Alex Zhang [00:19:27]: 这不是一个学术项目。

Swyx [00:19:28]: 是的。

Alex Zhang [00:19:28]: 我想提出这一点，因为这也发生在 RLM 上，而且它发生在许多其他工作上。就像，事情被过度炒作了，对吧？在某种程度上，某样东西被过度炒作，然后人们就说，“为什么这个被过度炒作了？”就像，“这很琐碎。这很愚蠢。”我在 Jev 上也看到了同样的情况，因为我认为发布时就像。有这一整套关于，比如，学术界的，或者他们不是学术团体，但是，比如，人们必须做品牌推广，他们必须，比如，某种程度上营销他们的研究。所以，比如，我理解，但我认为有很多讨论关于 Jev 只是，比如，我们多年来已经知道的东西。我认为这有点错过了重点，比如，为什么这样一个系统如此有趣？为什么它不只是某个愚蠢的 NLP 分类器，比如，我们在，我们在机器学习入门课上一直在做的？我认为 Jev 真正有趣的地方在于，它开启了这个问题：语言模型是正确的吗？比如，以它们现有的形式，我们能否考虑一个不同于文本到文本的设计空间？因为他们所做的基本上是，他们说，“我会利用这个语言模型主干。比如，我知道它捕捉了大量关于语言的信息，但我要改变模型的输出空间，给你一个权衡，那就是我会在……上进行非常快的推理。”比如，如果你对这个问题有一些先验，比如，假设我只需要做一个二元分类。我会让我的语言模型做这个并支付，比如，400 倍的成本吗？比如，不。那——这，比如，很傻，对吧？而且我认为在很长一段时间里，因为实验室是唯一控制的地方，你永远不会使用除了，比如，GPT-4 或 GPT-6 或 Fable 之外的东西，因为它们是最好的模型。但正因为如此，比如，人们已经有点习惯于这个想法：语言模型只是一个自回归解码器。比如，我们已经接受了这一点。我认为当 RLM 出现时，也是一样。比如，其中一条评论，比如我得到的一个非常频繁的批评是，“这不是一个语言模型。”或者比如，“当我看到这个时，比如，我以为这是一个新架构，但实际上不是。”而我对这个的回应是，比如，“好吧，语言模型只是建模语言。它不必是这个 transformer 解码器，”？而 Jev 真的很有趣，因为，比如，我们现在有了一个新的

Alex Zhang [00:21:49]: 需要调优的东西，比如输出空间是什么，以及这如何影响推理延迟？我认为我们实际上可以开始就语言模型本身的各个部分提出这个问题。我们在循环 Transformer 上也看到了这一点。类似的想法是，围绕它的很多关注都是，“这是个愚蠢的想法。”就像，“为什么？谁在乎这个？”但它是，这是一个简单的想法，但它实际上。它打开了一整套全新的问题，我认为，特别是如果你是一名博士生，这些就是你想回答的问题。因为我认为我们不知道。例如对于 Jev，我们不知道我们能把它推进到多远。对于循环 Transformer，我们也不知道我们能把它推进到多远。如果你只循环模型的一个子集呢？如果你路由到，比如，只有。比如你有一些路由器指向模型的不同部分？比如，你能在模型内部模仿你在 harness 中所做的事情吗？你能在 harness 选择和模型架构选择之间架起什么桥梁？这些都是我认为像这样的工作所打开的问题，而这，真的令人兴奋。特别是 Jev，当我看到它时，我就像，“这对 RLM 来说实际上真的很有用。”就像，我认为它，它。这说得通，因为 RLM 或 swarm 或类似系统最大的瓶颈是它们很慢。当你一直进行多次语言模型调用时，你没有正确地分配你的计算，因为，比如，也许有一些琐碎的事情你只想让一个简单的模型去做，但你做不到，因为你的语言模型就是这么一个笨重的东西？所以我非常兴奋。我认为我们将开始看到超越标准前沿模型的新类型模型出现，而那就像。有了这些，比如，新的权衡，你可以做太多事情了。

Swyx [00:23:34]: 我还要提一下 Thinky 及其交互模型。

Alex Zhang [00:23:36]: 是的。对。另一个很好的例子。

Swyx [00:23:38]: 是的。所以，基本上就是试图打破从序列到序列、仅解码器的范式，就真的去做任何其他事情。

Alex Zhang [00:23:46]: 是的。

Vibhu [00:23:46]: 我认为有一种，有一种程度，如果你试图竞争，你不会去和一个做自回归的前沿实验室竞争

Alex Zhang [00:23:54]: 不。

Vibhu [00:23:54]: 解码器继续。比如，他们投入的计算扩展资源量，即使 Thinky 也不会。好吧，可能有一小部分能做到，但至少在那方面你实际上做不了太多。

Alex Zhang [00:24:06]: 我对 Thinky 了解不多，或者我也不想说什么，但就像如果他们的策略只是复制 OpenAI 或 Anthropic，那是个糟糕的策略。

Alex Zhang [00:24:15]: 因为，嗯，因为，比如，他们就是没有。比如，你有点必须从，比如，你有什么优势的角度来思考。如果你要使用相同的设置。我确信他们不是，但就像如果你要做相同的设置，你基本上是在你能控制的东西上竞争，也就是数据和计算，显然他们无法在那方面与前沿实验室竞争。所以是的，这说得通，比如，如果你是一家新实验室，比如。实际上，我不知道你是否会把他们视为一家新实验室，但我猜，比如

Vibhu [00:24:40]: 是的。这就是为什么他们在那里。

Alex Zhang [00:24:42]: 我觉得他们有点奇怪，是的。

Vibhu [00:24:43]: 他们在他们的名单上。他们发布了 Inkling。就像，他们不能。

Alex Zhang [00:24:46]: 除了 OpenAI 或 Anthropic，也许像 Meta 和 GDM，你就得做点别的？就像，只是。这是可悲的现实，但我认为。我实际上认为这是件好事。我很高兴，比如，规模效应起作用，这些公司会继续这样做，因为它打开了，可能的新玩家，如果他们发现了真正有趣的东西。因为我有点怀疑这是否真的在 Frontier Labs 认真进行，因为就像你为什么要那样做？比如，为什么你要冒险将大量计算资源分配给新的赌注，当旧的赌注已经有效时？所以

Vibhu [00:25:24]: 我认为这就是催生很多新实验室的原因，对吧？

Alex Zhang [00:25:27]: 是的。

Vibhu [00:25:27]: 你有一个副业赌注，你得不到计算资源，你就说

Alex Zhang [00:25:30]: 是的

Vibhu [00:25:30]: “好吧，我去，我去做那个。”

Alex Zhang [00:25:31]: 没错。

Vibhu [00:25:31]: 而且，你提到的潜在上行空间的例子就像 Jev，它便宜 X 百倍，出来了，也许它是语言模型。

校准、快速分类和新模型权衡

Alex Zhang [00:25:41]: 是的。

Vibhu [00:25:41]: 在这种情况下，它只是不同。

Alex Zhang [00:25:42]: 是的。

Swyx [00:25:43]: 是的。

Swyx [00:25:44]: 所以不猜测 Jev 到底是什么？

Alex Zhang [00:25:46]: 我想我有一些猜测它可能是什么。我见过一些人说，“哦，它就像扩散的东西。”我猜那

Swyx [00:25:56]: 那就是并行解码，对吧？

Alex Zhang [00:25:58]: 是的，并行解码。老实说，我认为不管它实际上是什么，因为我认为你可以。我见过一些，比如，它的开源复现。他们所做的真正让我兴奋的是，我不完全确定他们的优化目标是什么以及他们如何训练它。我认为，就像，这是我们也在思考的 RLM 的事情，就像，好吧，RLM 是一个非常简单的想法。如果我发表这篇论文，现在任何人都可以使用它。但区分 RLM 实际价值的是你是否能正确训练它，以及你是否可能围绕系统塑造一些架构来让它变得非常好。这是，比如，我积极在做的事情，我想。但我认为对他们来说，比如，他们找到了一种训练系统的方法，这完全不是 trivial 的。比如，我实际上真的不知道他们是怎么做到的。我在网上看到一些比较，有些人声称他们使用了 Qwen，或者他们在 Qwen 之上进行了后训练，但你使用的每个开源 Qwen 都会更差，因为他们训练它所做的任何事情显然非常有效。所以这，这非常令人兴奋。

Swyx [00:27:04]: 有一个校准的要素，这是一个罕见的话题，我认为人们甚至不知道或不理解。我们在与 Hugging Face 的 Clementine Foley 的对话中讨论过，她曾经负责 Hugging Face 的评估，基本上就是模型倾向于给出最可能的下一个 token。但是，当你问它们“你有多自信？”时，它们会对你撒谎，因为它们只会给你最可能的下一个答案，而不是真正地校准。比如，我实际上有百分之五十的把握，或者我有百分之二十的把握，让我们试着校准一下。我想说，如果有的话，我认为实际上生成合成数据是相当容易的，因为你可以看到真相，然后合成生成一堆答案，让 Jev 分类，然后与真实情况比较。

Alex Zhang [00:27:52]: 哦，我明白了。

Swyx [00:27:52]: 那将是我对此的逆向工程。

Alex Zhang [00:27:54]: 是的。

Swyx [00:27:54]: 我实际上。我认为校准可能是被低估的。就像，人们只是把它当作一个非常快的分类器，但实际上他们甚至没有使用概率或校准估计。

Alex Zhang [00:28:04]: 是的。

Vibhu [00:28:04]: 我认为它仍然被误解了，重申一下。当你问一个模型“你有多自信？”时，它会吐出什么，百分之四十三。大的区别在于这是一个有根据的分类，对吧？

Alex Zhang [00:28:16]: 是的。是的，我非常兴奋地看到人们用这个模型做什么。它能解决所有问题吗？不，当然不能。但我认为它解决了一类我们传统上难以解决的问题，那就是低延迟的事情。所以我喜欢游戏方面的例子。那实际上就像。我

Swyx [00:28:34]: 是的，Doom 的例子

Alex Zhang [00:28:35]: 是的

Swyx [00:28:35]: 非常好。

Alex Zhang [00:28:35]: 我有一个关于语言模型玩电子游戏的基准测试。我一直着迷于你是否能让一个智能系统玩新游戏和类似的事情。所以我认为他们有一种独特的方式来做这件事真的很酷，以捕捉语言和理解，在一个快速的、非常快的模型中。

语言模型玩电子游戏

Vibhu [00:28:59]: 哦，既然你在谈论这个话题，关于电子游戏你有什么想指出的吗？

Alex Zhang [00:29:02]: 哦，是的。

Vibhu [00:29:03]: 这是。你在任何项目上都做过基准测试，对吧？

Alex Zhang [00:29:05]: 所以，是的。我想这些数字已经非常过时了

Vibhu [00:29:08]: 是的。

Alex Zhang [00:29:08]: 因为很多模型已经非常不同了。我实际上看到有人运行。有一些人在这些游戏上运行更新的模型，这真的很酷。我想这个基准测试的一般前提是我们只是想看看视觉语言模型是否足够好，比如，在包含延迟约束的情况下接入游戏。因为这实际上。我是在 Claude Plays Pokémon 出来后不久提出这个的。

Alex Zhang [00:29:33]: 所以这大概是两年前的事了，我想现在这已经算是老古董了。但我觉得这套任务真正酷的地方在于，它们涵盖的游戏非常多样。而且，我认为大多数游戏都是人们知道或以前见过的游戏。我看到，是的，Jeff 在玩 Doom。我得说我不这么认为。我觉得他们只是在玩非常简单的关卡之类的。但说实话，大多数模型仍然做不到。或者我其实不认为有任何模型能真正有意义地解决这些游戏。比如有些游戏它们能解决。我想我见过 Astra 能解决 Kirby 游戏。而且，对于这个基准测试，我们有意设计了一个非常精简的 harness。我稍后会谈到关于 harness 的这一点，因为我认为有一整场对话可以展开，比如 harness 的价值是什么？比如，harness 对模型来说到底有什么目的？但总的来说，我觉得是的，我希望很快或不久就能看到所有这些游戏被更新的模型击败。

Vibhu [00:30:35]: 是的，这很有意思。就像旧的 Cloud Place Pokemon，它们会从 RAM 读取状态，看看哪些 tile 可以走之类的。我们很久以前和他们做过一期播客。

Alex Zhang [00:30:45]: 明白了。

Vibhu [00:30:46]: 是的。就是好玩。

Swyx [00:30:46]: 是的，而且类似。就像 Jeff 没有视觉

Alex Zhang [00:30:48]: 是的

Swyx [00:30:48]: 所以你得输入，

Harnesses as Compositional Generalizers

Alex Zhang [00:30:50]: 是的

Swyx [00:30:50]: 这些，比如游戏状态和所有这些。我们直接进入 harness 的部分吧

Alex Zhang [00:30:54]: 太好了

Swyx [00:30:54]: 因为你提到了。语言模型 harness 是组合泛化器。

Alex Zhang [00:30:59]: 是的。

Vibhu [00:31:00]: 你读那个读得很费劲。

Alex Zhang [00:31:01]: 解释一下。是的。好的。所以我对人们如何看待 harness 一直有点不满意，因为人们会这样比较：“哦，我喜欢 Claude Code，我喜欢 Codex，我喜欢 Pi。” 就像，“不，我喜欢 Oh My Pi，我喜欢 Prime Agent。” 说实话，我觉得它们都一样。这些 harness 的大多数设计决策或者设计选择都是一样的。也许 Prime Agent 有点不同，因为它本质上是一个 RLM。但总的来说，我觉得我们可以在 harness 上更有创造力。我的意思是，如果我们从 harness 到底为模型做了什么这个角度来看。基本上，当你试图解决一个问题，并且想用语言模型来解决它，比如一个非常困难的任务时，我们发现的一件事是，下一 token 预测对于完成很多这类任务来说是一种非常别扭的形式。例如，拿 SWE-bench 来说。当你在浏览一个代码库时，你能通过一次语言模型调用就搞清楚如何完成所有这些吗？就像你只说，“解决代码”，或者“在我的代码库上解决我的查询”。不能。所以我们依赖 harness 来帮你做这类事情。我觉得有趣的是，harness 是一个非常有主见的程序，关于你想如何让语言模型适配一个问题。我提到这个是因为，当我们思考 harness 在做什么时，我们真的应该思考，harness 中的哪些选择让语言模型能够解决这个任务？以及我能不能实际上就让一个语言模型来做这件事？因为如果你想一想，既然循环 transformer 已经出现了，我觉得真正有趣的是，你可以在某些方面把循环 transformer 建模成类似 harness 的东西，对吧？你只是在模型上循环。现在你可以说，“哦，我不是在解码”，所以这有点不同。但总的来说，我们不知为何一直固守着同样的模型架构选择。而且我。关于为什么有很多论点，但很明显，我们现在是围绕 harness rollout 来训练模型的。所以有一种非常别扭的基于 harness 做训练的方式，就是我们训练一个语言模型在 harness 内行动，但现在这就像是一个很长的、也许是多智能体 rollout。而且有一些非常别扭、取巧的做法。所以这篇博客讲的是，嗯，你可以这样思考这里发生了什么：如果 harness 基本上是在帮助模型解决某个特定任务，那么不同的 harness 设计选择是否真的能做一些更有意义的事情，而不仅仅是“这里有一些能帮你的工具调用。这是一种在你的代码库里 grep 的方式。” 所以这实际上。这篇博客的想法也是随着 RLM 的想法一起出现的。我们只是没有那样包装它。我觉得这实际上是真的。关于 RLM 还有很多其他想法，我们会陆续发布，但它们从一开始就都在那里。这些都是围绕设计决策的。我觉得 RLM 让我喜欢的地方在于，我感兴趣的不同抽象有很多次迭代和版本，最终 RLM 最合理。但有很多不公开的原因导致情况如此。你会看到的。所以在这个例子中，我们在 RLM 上看到的一件事是，如果你充分地把上下文卸载出去，并让模型在这个上下文上写代码，你会得到一个非常奇怪但有用的性质，那就是当模型在训练中认识到如何解决一个任务时，结果发现许多任务的解决方案非常相似，甚至在你都不。甚至你都不清楚这些解决方案是相似的。所以在这个例子中，我们有一个检索任务，还有一个聚合任务，它们的查询非常不同。领域完全不同。当你在这两个任务上训练一个常规语言模型时，轨迹看起来非常不同。所以当你。比如，假设你用 Pi 或 Claude Code 之类的，这不在博客里，但我们确实有这些结果。你会发现，这些 harness 在这些问题之间区分得太多，尽管解决方案是相同的。所以我们在训练 RLM 时发现的一件事是，当它看到这些问题时，是一样的。它把这些间题看作相同的原因是，子智能体看到的是不同的问题，但子智能体在解决一个更容易的子任务，所以你有信心它足够聪明能完成。但对于基础的整体策略，它们最终看起来是一样的。所以当你训练左边的任务时，例如，它可以立即解决右边的任务。所以如果你看我们有的那些图，你会观察到的一件事是，当你只是天真地在这些任务上训练你的 RLM 时，它们自然会学会泛化，例如，到更长的任务，因为策略基本上是一样的。你只是在修改一个长度变量。这实际上也适用于不同的任务，而且它们甚至不是——它们不是长度上的不同。它们是完全不同的任务，数学任务与写作任务。但解决方案，那个元层面的高层解决方案是一样的。所以当你在其中一个上训练 RLM 时，它会把这个行为泛化到第二个上。这里没有魔法。我想也许这是我想强调的一点。就像

Vibhu [00:36:38]: 你会怎么描述它们正在学习的东西？我觉得你在这里说的是，你在短任务上训练它，它们能泛化到长 8–30 倍的任务上。

Alex Zhang [00:36:47]: 是的。

Vibhu [00:36:48]: 它们是在学习如何解决这类问题，还是说，它们实际学到的核心东西是什么？

Alex Zhang [00:36:52]: 是的。它们是在学习如何解决特定长度的这类问题。而事实证明，当你采用它们学到的策略时，它可以直接迁移到更长的长度上。就像，它们

Vibhu [00:37:05]: 是的。

Alex Zhang [00:37:05]: 实际上是同一个程序。这一点非常令人兴奋，因为它所暗示的是，当你有一个用于训练模型的数据或环境语料库时，希望在于：第一，你可以用更少的环境进行训练，并泛化到比现有模型通过现有那种朴素训练所能做到的更多场景。但第二，你仍然希望利用你拥有的所有数据。所以当你在这些任务上训练时，希望它能泛化到更广泛的一类问题。而为什么这甚至更令人兴奋，至少在 RLM 或任何递归调用系统的语境下，是因为这个论证是归纳成立的。举个例子，因为我说过，我声称竞赛编程和 GPU 优化使用非常相似的技能组合。模型可以——harness 可能需要你以某种方式引导它，但它可以学会，比如，“好吧，我解决这个 GPU 编程任务的方式与我在竞赛编程中学到的非常相似。所以我要列出一组解决方案。我会生成子代理来列出有前景的解决方案，然后我会写这个循环来遍历并检查这些解决方案，也许进化它们，并针对验证器进化它们。”在这两个任务之间，这看起来是一样的。但子代理所做的事情可能是独特的、不同的。但甚至子代理所解决的问题可能实际上也是相同的形式，对吧？因为这是一个递归论证。所以我在整篇博文中想表达的是，我们应该重新思考 harness 的角色，因为 harness 实际上可以极大地提高模型的泛化能力以及它所获得的数据量。这并不限于 RLM。我认为还有一大类尚未被发现的 harness 实际上可以产生类似的特性。再把这个论证延伸一点，我认为你还可以从中推断出，如果我看一个 RLM，RLM 中哪些组件实际上是必要的，我是否可以直接训练一个模型来做这件事？我能否训练一个模型在其前向传播中隐式地充当 RLM？这是一个非常奇怪的想法，因为你可能会说，“哦，代码是不可微的，等等。”但我们将开始发现许多这种行为的近似。而且我认为，我们将看到的不仅仅是“我要设计一个新的编码 harness，使用某种特殊形式的压缩”之类。我认为我们可以在这里更有创造力。对于这些语言模型，我们可以做的事情太多了，我认为我们只是没有在做。我对此非常兴奋，因为我认为我们可以从非常有主见且良好的 harness 设计中获得重大收益，这种设计更适合扩展。这里的意思是，例如 RLM，是一种非常原始的归纳偏置。除了它与我们目前所做的非常不同之外，这个设计没有什么特别之处。但这可能随着我们可用的数据和环境而更好地扩展。

Vibhu [00:40:20]: 我想，人们可能会问的相反问题是，当前的 harness 非常偏向于编码，而人们看到这在很多领域都有效。Cloud code 正被用于设计、演示文稿

Alex Zhang [00:40:33]: 是的。

Vibhu [00:40:33]: 一切。MuseSpark、Grokbot。

Vibhu [00:40:36]: 这些是非常简单、不带倾向性的 harness，擅长代码，而且也在扩展。我想，我们如何改进它们的例子是什么？

Alex Zhang [00:40:48]: 是的。让我提另一篇论文，它最近刚出来。就像那篇 harness tax 论文。我想是 Arena 写的。我非常喜欢这篇论文，因为它提出了一个我原本就有的先验，基本上就是，大多数

Vibhu [00:41:07]: 所以它证实了这个先验。

Alex Zhang [00:41:08]: 是的。就像，大多数 harness 选择并不重要，因为

Vibhu [00:41:12]: 是的。

Alex Zhang [00:41:13]: 所有这些 harness 都是一样的。但我要说，比如 Grokbot，据我理解，实际上与这些其他 harness 的设计方式相当不同，我很喜欢这一点。而且我认为至少从这里可以清楚看出，就像，我很确定。Anthropic 或 OpenAI 只在它们自己的 harness 上训练。它们可能不会在竞争对手的 harness 上训练。我猜不会，因为我不知道它们为什么要那样做。但是

Vibhu [00:41:38]: 但是，这是你在开源模型中看到的事情，对吧？就像，Qwen 非常擅长使用 open code。

Alex Zhang [00:41:43]: 是的。

Vibhu [00:41:43]: 它们需要在 harness 中训练。旧的 Gemma 在这方面是出了名的差。

Alex Zhang [00:41:47]: 是的。

Vibhu [00:41:48]: 模型很好，但你需要在 harness 中训练。

Alex Zhang [00:41:50]：不过我觉得，随着模型变得更聪明，或者说变得更好，这种区分就没那么重要了，因为如果你把 Astra 放进 open code 里，它不会失控，因为我觉得它已经足够好了。我这么说是因为这些不同 harness 之间的唯一好处，大部分情况下只是成本。我想说的是，Sugru，如果你把这些模型接入 RLM，它们其实还是没那么好。它们还行。我认为这主要是因为，我们围绕训练的那类 harness 就是这一类 harness，这种 pi loop，这种——我喜欢称之为“把轨迹当作提示”，意思就是你把整个 rollout 的轨迹保留下来，作为主模型使用的上下文。即使你用了子代理，它基本上还是这种形式。我觉得我们会——如果我们想探索新的 harness，就需要有团队专门去做有意义的实验，比如扩展新的 harness，也许是在不同 harness 设计上做后训练扩展。我觉得我们确实可以从这类事情中获得非常有意义的知识或收益，不管是 RLM 还是别的东西。这很令人兴奋，因为我觉得，比如说，如果你在……Fable 在很长一段时间里是 RLM 的最佳模型，因为他们有动态工作流，而且很明显这是一种某种程度上被训练出来的能力。即使这个模型在 RLM harness 里还是有点笨，它的表现仍然比其他模型好得多。Astra 现在在做这些事情上也足够好了。但是

Swyx [00:43:33]：等等，这就是我们看到 Fable 最适合 RLM 的地方吗，还是有别的

Alex Zhang [00:43:38]：哦，不，这些我想都是内部结果。

Alex Zhang [00:43:40]：对，我没有，我没有这些

Swyx [00:43:41]：好的

Alex Zhang [00:43:42]: Public right now. But in general, like, I think you can, You can very easily tell that we have not optimized for RLM, like, workflows yet. And I think if we get models that do this correctly, they will be a lot more efficient as well. you can kind of just think through, like, why this is the case, right?

Vibhu [00:44:03]: I think this is the point where you have to give the ten-second what are RLMs.

What Is an RLM?

Alex Zhang [00:44:07]: Oh, yes.

Vibhu [00:44:07]: Because there’s a lot of listeners here that

Swyx [00:44:09]: Yes. We’re, we’re assuming a lot of knowledge.

Alex Zhang [00:44:10]: Yes.

Swyx [00:44:11]: Also, I think you. But you have set some context

Vibhu [00:44:13]: Yes

Swyx [00:44:13]: So you can. Like, with everything we just said

Vibhu [00:44:15]: Yes

Swyx [00:44:15]: Can we have a clean, crisp definition of RLMs?

Alex Zhang [00:44:18]: Yes. Okay. I want to go back to the blog, the

Vibhu [00:44:21]: Yes

Alex Zhang [00:44:21]: The compositional generalizers blog. This one. Okay.

Swyx [00:44:24]: Okay.

Alex Zhang [00:44:24]: This is, like, the best. I wish I had this in the original paper. An RLM is basically just a harness design where the only tool in the harness is code, which is this programmatic subagent calling thing, where it has the option to call itself as a tool, and it has other tools. But all of these things are functions in code, and the context that it’s dealing with is always stored in some memory inside of this code environment. So this could be a file system. Like, this could be, like. I’ll give you an example, Prime Agent. The trajectory of Prime Agent, like the context, even the. when you compact and do all these things, is stored on disk. So the model can always reference its original context, even if it’s compacted, and all of its tools are run inside of, let’s say, like, a Python REPL or a Bash REPL. And so this. it’s like this very primitive abstraction. And I would say, like, where most harnesses differ is, A, context offloading is not done that, like that. if you look at Prime Agent, by the way, Prime Agent does context offloading, but not all the way. So, like, it still maintains the standard cod code, Codex loop of, like, trajectory as a prompt where you compact, but it has the additional kind of, like, the context is offloaded, and it only has. The unique point of Prime Agent is that the only tool is IPython. So this is, like, the very kind of generic abstraction around, like, RLMs. Yeah.

Vibhu [00:45:59]: Concretely, what’s the core thing RLMs are trying to solve?

Long Context, Composition, and Locally In-Distribution Tasks

Alex Zhang [00:46:02]: Yeah.

Vibhu [00:46:02]: At one point, I think when it first came out, it was context.

Vibhu [00:46:06]: I’ll pass the question.

Alex Zhang [00:46:06]: Yeah. So the original problem was harnesses have a really bad time dealing with long context. They typically were used to only really do them for, like, specific things like code. Like, they could deal with your code base because it was trained on it. But now it’s more around what this blog is talking about, which is Compositionality and the fact that, like, I think harnesses. We want to have language model systems that have much more control over the actions they make at every step. And what by this is tool calls are very limited because you have to invoke them every turn. Like, you have to invoke tool A, then tool B, then tool C, and there’s no central context that you can kind of draw back from. And RLMs are specifically, like, designed around composition and having, like, a central context that you can always draw from. and this context is, like, designed around the existing language models. Another, like, very similar example actually in design is, like, agent swarms, for example, the Hugging Face incident. Like, these agent swarms have, like, a message board that they learn to communicate over. And this message board, in some sense, is the shared context that they, like, act over. And RLMs basically say that, like, the best way to communicate through this is in code. Like, you write the code to do this, and it’s, it’s because these models are so good at writing code. like, we wanna take advantage of that fact.

Swyx [00:47:37]: And then for this compositional thing, up to and including generating your own harness, specific for the task.

Alex Zhang [00:47:44]: Exactly.

Swyx [00:47:45]: Right?

Alex Zhang [00:47:46]: I think we will start to see that if you go up to, this figure. Okay. So we talk about this idea of locally in-distribution tasks for a harness, and it is like a, an idea on top of, like, in-distribution tasks. When we think about language models, an in-distribution task is just a task where, like, the prompt is something that the model has either seen before or, like, has seen some version of it. Most harnesses work like the one on the right, which is they keep appending the trajectory as a prompt. And so eventually, unless you’re Anthropic or OpenAI and you train on, like, kind of these, like, user trajectories, most of these things end up being out of distribution for the most part. But locally in distribution is basically the compositional argument of if an RLM breaks down its computation into, like, kind of a meta-harness of sorts or, like, a program that involves subagents that, like, look at a local problem, every individual language model call over the course of this task is in distribution, even if the entire task itself is out of distribution. And this is a very desirable property, I think, for obvious reasons. Like, if every task is in distribution for each individual language model call, you will probably get to the right answer. so.

Swyx [00:49:00]: The logical limit of RLMs is RLLMs where, like, you not just, you don’t just write the harness, you also train a custom model for

Training RLMs and Smarter Harnesses

Alex Zhang [00:49:11]: Exactly.

Swyx [00:49:11]: You collect data, everything.

Alex Zhang [00:49:14]: Yeah.

Swyx [00:49:14]: Like, it’s a fully automated AI researcher inside of your harness.

Alex Zhang [00:49:16]: Yeah. We will see where the training of RLMs goes. I will say, as an academic, I am not working on this at MIT, or at least in the scaled sense, because I can’t afford to. but there are companies out there that are working on this. I think Prime and Select is very clearly working on this, and it’s very cool. Like, I’m, I’m very excited to see. Maybe we’ll observe, I don’t know, but maybe we’ll observe better, like, post-training scaling laws with when you train around a smart harness. Maybe we’ll even see smarter harnesses that come out and, like, they work better around these kinds of principles.

Swyx [00:49:50]: What is a smarter harness? Like, that doesn’t mean anything. You just said they’re all the same.

Alex Zhang [00:49:54]: No. What, more of what is, like, Claude Code, Codex, Pi, et cetera, are all the same in that, like, when you break down the logic of the harness, it’s, like, virtually the same thing.

Swyx [00:50:06]: Yeah, two calls in a loop or

Alex Zhang [00:50:07]: Yeah

Swyx [00:50:08]: Whatever.

Alex Zhang [00:50:08]: But With RLMs and with other harness abstractions, it looks very different. And this is where I think you really distinguish. it’s, it’s in the same way that, like, I think with language model architecture choices, a lot of architecture choices end up kind of looking the same when you, like, scale it out or, like, it. The differences end up being, like, somewhat minor in terms of. for a lab it’s not minor, but, maybe one model converges better than the other one, like, slightly. But in general, like, if you were. So for example, pre-training scaling laws only hold because the architecture choices we have are somewhat stable right now. But if you were to completely change the architecture, pre-training scaling laws probably don’t hold. Or, like, these kinds of. This, like, power law is gonna look very different. And it’s like the same thing with harnesses. Like, I think all the harnesses we have right now, for the most part, roughly look the same, but there are some exceptions to this, I think, that are coming out.

Swyx [00:51:06]: I was gonna say, I actually, one of the things that I’ve been more interested by, like, talking about PhD students who take big risks, is that people have been. People also pursuing the other side, which is pre-training scaling laws don’t hold if you change data. right now it’s just raw, unstructured text, corpus of internet. What if you had a better data representation to train on? Then yes, your scaling law would change as well.

Alex Zhang [00:51:28]: Yeah.

Swyx [00:51:28]: So there’s architecture, there’s data, and, whatever else, you can think about. Well, so I just wanna get back to this. it all makes sense. It’s, it’s very interesting how you sort of recurse up and down the stack from, like, very conceptual to, like, not like, well, this is where we are today.

Prime Agent and Opinionated Harness Design

Swyx [00:51:43]: But, like, yeah, obviously, it can scale up and down. I guess, I’m curious, how did you start working with Prime? Is Prime taking on more work with this? Is this their answer to Hermes agents?

Alex Zhang [00:51:56]: Mm.

Swyx [00:51:56]: You mentioned Grokbot is a little bit different. I just wanted to, like, namecheck all these guys and get your thoughts on each.

Alex Zhang [00:52:01]: Yeah. I got involved with Prime, after they released a blog post, by the way, not affiliated with me at all, about, like, how they believed RLMs were kind of the future. And I had a friend that was working there, GPU Mode, Matei. Like, we got in touch, and I think I agreed with a lot of the researchers there and, like, what they believed about harness design. Like, I was very impressed, I think, that, like, they understood the purpose of the RLM paper, which is not necessarily just to say that, like, we’re solving long context tasks, but actually, like, we want more opinionated harness designs.

Swyx [00:52:39]: Yeah. There’s always, like, the result of the paper that you choose to highlight

Alex Zhang [00:52:42]: Yes

Swyx [00:52:43]: Versus the actual point.

Alex Zhang [00:52:44]: Yes. As I would love to talk about, like, the incentives of academia and, like, the things around, like, why it’s kind of flawed and all the issues, and we’ll get back to that. Yeah. So anyways, I love the guys at Prime. So we kind of had been. After we decided to work together, we decided to look into training in RLM and also build this kind of RLM harness and kinda see where we can take it. That is how, like, Prime Agent came about, and I think the reception for Prime Agent has been pretty good. Like, the one thing I was worried about with Prime Agent is that none of them, at least at the time when we were building it, none of the models were that good at doing RLM stuff. So this was, like, pre-Fable, pre-Astra.

Swyx [00:53:27]: I guess, I think to take a step back, can you explain what Prime Agent is, how it’s different than

Alex Zhang [00:53:32]: Yeah

Swyx [00:53:33]: A traditional, Claude Code, what people would expect harness?

Alex Zhang [00:53:36]: Yes. So Prime Agent, I think I mentioned this a little bit earlier

Swyx [00:53:40]: Yeah, there was the diagram. Yeah

Alex Zhang [00:53:41]: Is basically. it is a. A harness on top of Pi, like Pi Mono, which is-- Pi Mono, for context, is like the, like a

Swyx [00:53:50]: Core agent

Alex Zhang [00:53:51]: A minimalist

Swyx [00:53:51]: Yeah.

Alex Zhang [00:53:52]: Yeah, like harness. I use Pi as the reference for everything because I think all other harnesses are basically just Pi.

Alex Zhang [00:53:57]: But it is Pi, except we explicitly restrict IPython to be the only tool that’s available to it. Every other tool gets loaded in as, like, a Python module, or like a Bash kind of script that it can run. So it uses the core RLM abstraction on top of Pi, and then it also has this continual harness thing, which is, Seth, he’s another PhD student. This is a thing that he used to get language model harnesses to play games. Like, so he worked a lot with Joel, who is the, like, Gemini plays Pokemon guy. And continual harness is also, by the way, very simple. I quite like it. It basically is this design, principle around, like, what parts of the harness can you let the harness itself modify? There are certain pieces that, like, you’ll let it modify its own skills, the subagents available to it, what the system prompt to the model is. And continual harness is available basically as a tool inside of the IPython kernel. And so that’s what Prime Agent is, like, how, what it’s designed around. Everything else in Prime Agent is like

Swyx [00:55:05]: Standard.

Alex Zhang [00:55:06]: Standard.

Swyx [00:55:06]: Standard.

Alex Zhang [00:55:06]: Right? Yeah. I think what is, what I really liked about it, and we got kind of lucky, is that, like, a lot of the new frontier models actually work really well inside. And actually, even a lot of the open-source models work really well, at least some of the newer ones. And there is another thing in Prime Agent I should highlight, which is that, like, we have a very particular agent-to-agent communication system or, like, framework, which is because RLMs tend to spawn many subagents, we want a way for subagents to communicate with maybe the root or with each other. And so there are some design decisions around, like, what each subagent is allowed to talk to, how it does it. Again, everything is in code, so it writes the code to do this kind of communication, which I think is really cool. And then there’s, I guess, persistent subagents is another thing that was kind of added, which is the subagents, they can last beyond, like, the standard runtime of the actual, like, original agent. And you can go into that subagent, you can prompt it more, like, you have more visibility and flexibility into what is kind of going on.

Swyx [00:56:10]: This is my number one pain with Codex right now. They, their subagents are just very ephemeral And they actively discourage you from using it for long-running things.

Alex Zhang [00:56:17]: Yeah. Yeah. Which I think it makes sense. Yeah. I

Swyx [00:56:21]: So the trick is just externalize to a file system, right?

Alex Zhang [00:56:24]: Yes. Yeah. That’s

Swyx [00:56:25]: Like, that’s the trick.

Alex Zhang [00:56:25]: That is the big trick.

Alex Zhang [00:56:27]: Yeah.

Swyx [00:56:27]: And well, and also, like, force everything to run through code. trust the model

Alex Zhang [00:56:30]: Yep

Swyx [00:56:30]: That can write code, and it’s gonna write its own harnesses itself. So is Prime gonna take on, like, training, post-training custom models for this? Is this a one-off collaboration between you guys, that’s it? Like, what’s

Alex Zhang [00:56:41]: Yeah. They are training a model, intern-- I think they were pretty public about this actually

Swyx [00:56:46]: Yeah

Alex Zhang [00:56:46]: Back in March.

Swyx [00:56:47]: Clearly it is their business.

Alex Zhang [00:56:49]: Yeah.

Swyx [00:56:49]: Yeah, so.

Alex Zhang [00:56:50]: Yeah. they’re, they’re showing that they can train it on their kind of hosted training stack. But no, so for model training, I’m, I’m not involved with them on that. The main reason is just I have other things in the PhD I wanna work on. I think, like, there are many other big bets to take,

Swyx [00:57:03]: Ooh

Alex Zhang [00:57:03]: Outside of just RLMs. some

Swyx [00:57:06]: Ooh

Alex Zhang [00:57:06]: Some I don’t know how much I can share yet. but in general, like, I think, I actually think one of the luxuries of being a PhD student, genuinely, is that there’s so many big bets to take. most of them will probably yield nothing, but it’s a really exciting time to be in research, especially because I think most progress in the field has been a little bit boring. Like, I’m not saying the outcome is boring, but the process of doing these things tends to be quite boring. and so there is kind of this question of, like, what do we wanna do next? but

Swyx [00:57:39]: Yeah

Alex Zhang [00:57:39]: We can talk about that later.

Third-Party RLM Work: Harvey, Headlong, DSPy, and ARC-AGI-3

Swyx [00:57:41]: Yeah.

Alex Zhang [00:57:41]: Yeah.

Swyx [00:57:41]: Okay, I wanna close out a little bit more of your research, and then we can,

Alex Zhang [00:57:44]: Cool. Yep

Swyx [00:57:44]: Start putting it out. since you released RLM, a lot of excitement about it. Any secondary third-party work that you wanna shout out as, like, that you guys should take a look at this?

Alex Zhang [00:57:53]: Oh, yeah. So, Harvey, the legal AI company, released a blog post, not affiliated at all, but they post-trained an RLM on their, like, legal work, which often involves a lot of, like, sifting through documents and kind of looking through, like, a variety of, specific information that maybe is not so easy to retrieve with, like, a pure retrieval system. And they show, like, really good results. It’s very exciting. I was shocked that they worked on this. They did not tell me, so when this came out, I was like, “Oh, that’s awesome.” So there’s this one I think is super cool and what they’re doing there. I think this is a collaboration with Base 10, by the way, as well.

Swyx [00:58:33]: Yes, this was Base 10.

Alex Zhang [00:58:34]: Headlong, which is law, the Law Institute’s kind of. it is their, like, persistently running harness. it’s very cool that

Swyx [00:58:42]: Oh, they renamed it? They used to call it something else.

Alex Zhang [00:58:45]: It was like Auto

Swyx [00:58:46]: Terminus.

Alex Zhang [00:58:46]: Yeah, I know. They’ve gone through. Yeah.

Swyx [00:58:49]: All right.

Alex Zhang [00:58:49]: So this is Andy Konwinski’s big project. it’s super cool. I love Andy. I don’t want to downplay what they’re doing because they’re using the RLM abstraction, but they’re doing something much cooler than the RLM, which is like they have a system that kind of what they call, like, thinks persistently. So even when you don’t query it has a way to, think through problems that it has in its context.

Swyx [00:59:16]: Oh, so it’s just like a always-on type thing.

Alex Zhang [00:59:18]: It’s like an always-on thing, but it’s, like, not that expensive. Like, they control the token costs, to make sure it’s not, like, burning through all your credits. This is super cool. I’m trying to think. There are many. Actually, if you go to the RLM, GitHub page, there’s a bunch of things I’ve linked, below. There’s a ton of really cool kind of things that people have been doing. Axe is another really cool one that I think it’s just by this one guy. It’s like a harness around DSPy and RLMs. DSPy also has an RLM. Oh, the last thing I’ll shout out is on ARC-AGI-3, I believe, there were a lot harnesses on their, like, Kaggle competition, like the official one, not the, like, public primates, like one that, or like what people have evaluated on. They all, like, claim to use or they reference, like TUFA, for example, some form or some inspired form of the RLN abstraction in their harness, which is really cool. I think it’s, This is where-- this is exactly the setting where you would see a lot of benefits from composition and using code and combining, like, neuro symbolic systems with AI. And so

Swyx [01:00:27]: Yeah.

Alex Zhang [01:00:27]: Yeah. Very cool.

Swyx [01:00:28]: We love a good neuro symbolic reference.

Agent Swarms, Unsolved Math, and What the User Should See

Alex Zhang [01:00:30]: Yeah.

Swyx [01:00:30]: You also, mentioning ARC-AGI-3, OpenAI comes out and says, “We’re at 99.9% on this.”

Alex Zhang [01:00:36]: Yep.

Swyx [01:00:37]: They also say, “We solved Navier–Stokes. We just threw a model at it.”

Swyx [01:00:40]: There’s some debate around whether or not it’s just model.

Swyx [01:00:44]: Are they using an RLM? Do?

Alex Zhang [01:00:46]: I would guess probably not, unless you say, like. I’ll, I’ll be, I’ll be careful here because, people debate what is an RLM, what is not an RLM. It’s somewhat clear that what they used is some kind of swarm of agents with a shared, some shared context, like some shared file system. And, like, this is very much in the spirit of RLM stuff, but I think there’s a, there’s a lot of, like, more clever things that they did that’s not maybe related to the RLM itself. I agree a little bit with the idea that, like, a harness is not that necessary for what they did. The way that I would put this is that I think a model, like a GPT-6 Astra type thing, is technically smart enough, conditioned on the right information, to come up with a proof for these very difficult problems. Now, how you get to that information is a giant question mark. And in their case, it probably came down to, like, a very long search over, like, many of these sub-age-- or many of these, like, agents in the swarm and maybe also, like, researchers cond-- I’m, I’m actually not sure about this part, but putting in, like, their kind of intuition as to, like, what you should explore and things like this. And ultimately, like, this produced some information that some agent was able to take to finish the proof. And so in that sense, like, I think, was the harness that important? No. And I think what this is pointing at is, like, the specific details of a harness do not really matter, and I think that’s also what that, what the harness task paper is pointing at, which is that, like, beyond the user’s feeling of the harness, realistically all that matters is just, like, how are you composing these agents in a meaningful way to get to the final answer? And maybe that’s what, like, swarms and all these things are really about. And so from my POV at least, if we start to think about, like, for user use cases, what do we want out of harnesses and things like this? Like,

Alex Zhang [01:02:51]: We want to take the good parts out of these, like, the Claude codes, the Codexes, like the stream that people like to see. But, like, under the hood, whatever is running can be some really weird, complicated swarm of agents that, like, ultimately come up with an answer. The user doesn’t wanna see that, though, obviously, right? Like, it’s, it’s not legible information. And so I-- this was another kind of thing in the spirit of RLMs, like recursive language model. It sounds like it’s a language model and, but it’s not a language model architecture. But the reason for this is, like, I think we will start to see in the future probably one day, and I wrote a blog about this, what we think of as a language model, like the thing that we query, might actually be like a swarm or like a scaffold or like some Weird harness design that scales very well, but the user just doesn’t see it. Ultimately, all the user sees is some front-end version of this harness. And yeah, I think it’s a relatively safe bet at least to make that this is what we will see.

Swyx [01:03:48]: Yeah.

Alex Zhang [01:03:49]: And this maybe goes back to the limitations of the base transformer. Like, obviously if you just took a base transformer and you said, like, “Solve Navier–Stokes,” or something, it’s not gonna do it. Like, yeah, we all know this is not what’s gonna happen. But yeah, I think this is maybe the more interesting part. and maybe the claims around, like, did the harness matter is more around this, of, like, just arbitrarily pointing models, like, or agents at a growing kind of context of information maybe is just enough to solve very difficult problems. that I can buy.

Swyx [01:04:20]: While there’s a lot in there, I do wanna say, the amount that OpenAI spent is semi-public. It’s, 10,000 agents in 88 hours.

Alex Zhang [01:04:29]: Oh, yeah.

Swyx [01:04:29]: 130 billion output tokens, which is estimated to be about 40 million dollars in public pricing.

Alex Zhang [01:04:34]: Surprisingly, actually, like, less than I thought.

Swyx [01:04:37]: Yeah, not that much.

Alex Zhang [01:04:38]: Yeah. Yeah.

Vibhu [01:04:38]: 130 billion output for the final, but as you said, there was a lot of context being passed around.

Vibhu [01:04:44]: It’s more than double that in just the total agent messages being sent.

Alex Zhang [01:04:47]: Yeah.

Swyx [01:04:48]: Yeah.

Alex Zhang [01:04:48]: Yeah. Yeah.

Swyx [01:04:49]: I think, one thing I was honored to bring up also was Cursor as far as, like, swarm stuff is concerned.

Swarm Architectures, Coordination, and Token Efficiency

Swyx [01:04:53]: This is slightly older, meaning February, which is ancient.

Alex Zhang [01:04:57]: Whoa.

Swyx [01:04:57]: But if you scroll down all the way to the final sort of multi-agent architecture that they arrived at, it was basically an org chart of a normal software team. One thing I’m thinking about, because I basically. There’s, like, this, gather all function That you have to do with subagents or. It’s very similar to GPU programming, actually.

Alex Zhang [01:05:16]: Yeah. Yeah. Yeah.

Swyx [01:05:17]: And so, that’s a bottleneck. This is a bottleneck.

Swyx [01:05:21]: If there’s one main guy, that’s coordinating all the sub guys, then they have to, like, gather again and then re-coordinate.

Swyx [01:05:29]: That’s slow. That’s, that’s crappy. what a true swarm should be is everyone is just their own person.

Alex Zhang [01:05:35]: Yeah. Yeah.

Swyx [01:05:36]: Right?

Alex Zhang [01:05:36]: Well, I agree with this, and I think that there is a question to be had, though. Let me give an analogy, which is like, when would you use compaction and when would you use an RLM? And there are many settings where an RLM can solve maybe a more difficult task than compaction can, but you would prefer to use compaction in most cases because it’s cheaper and it’s quicker. And I think in the context of agent swarms, there is a similar thing going on of, like, I’m. Fairly certain that, like, 95% of the swarm is entirely useless, or, like, what it’s exploring is entirely u-- You’re just burning tokens. Versus in this setup, maybe not so much. I’m not sure. maybe it’s also the case here.

Swyx [01:06:17]: Everyone has a job. This is your board.

Alex Zhang [01:06:19]: Yeah.

Vibhu [01:06:19]: I think at some level that’s how problems are framed, right?

Alex Zhang [01:06:23]: Yes.

Vibhu [01:06:23]: So, like, if you have a search problem and you’re spanning out a bunch of subagents to do search, there’s gonna be a lot of useless information, right? There is one retrieval answer That you’re getting and you’re spanning off, but that is consciously understood, right?

Alex Zhang [01:06:36]: Yes. But there is kind of this question of, like, what is appropriate to solve for which problems? Like, what design-- in theory, OpenAI can use-- can package up this API and they’ll call it swarms, and then they’ll give it to you and they’ll be like, “Point this at any problem and we’ll give you a solution.” But maybe

Swyx [01:06:53]: Yeah, it’s called, it’s called pro, right?

Alex Zhang [01:06:54]: Yeah. maybe you’ll have to pay like 40 million dollars to get a result.

Alex Zhang [01:06:57]: And it’s like, well-- But it’s exciting. I will say, like, it is-- it’s very exciting that we even have the option to point 40 million dollars at a problem and solve it.

Vibhu [01:07:07]: Yes.

Alex Zhang [01:07:08]: But there is still kind of, a lot of research to be done in this area around, like, what is necessary. Like, what do we want to do? What design do we want? we probably don’t want everything to be a swarm, but, like, where do we draw the line? Like, can the agent design that or decide that for itself? Et cetera. So.

Open-Endedness and Research Without a Fixed Objective

Swyx [01:07:26]: Yeah. and then, just a quick check in case you have opinions on this. Have you looked much into open-endedness as a general category of problems?

Alex Zhang [01:07:35]: Oh

Swyx [01:07:35]: Meaning no prompts, just go.

Alex Zhang [01:07:38]: A little bit. so I was at Sakana for a summer, right after, or I guess right before my PhD, and that’s something that they work on a lot there. And I think there’s a lot of people even at, like, Recursive Super Intel-- There’s many of them now.

Swyx [01:07:53]: Yes, we just had Richard Socher on.

Alex Zhang [01:07:55]: Oh, yes. Yeah. So, like, Richard’s company and then also. Actually, wait, that might be the s-- it might be the same company. I don’t remember. Is Tim Lautenschlager also

Swyx [01:08:03]: Yeah.

Alex Zhang [01:08:03]: Okay. Yes, that company.

Swyx [01:08:04]: He’s the main co-founder. He used to be head of open-endedness for Google.

Alex Zhang [01:08:07]: Yeah.

Swyx [01:08:07]: Yeah.

Alex Zhang [01:08:08]: So I think with open-endedness problems, like, I view them as somewhat similar to even, like, unsolved math problems. Maybe that’s a weird way of framing it, but, like, I think a lot of the techniques in terms of, like, how people like, approach them are kind of the same. Like, evolutionary search is, like, very similar to just launching swarms of agents and hoping that, like, they come up with, like, an interesting s-- And this is what, like, AlphaEvolve and some of these other works did, like a year or two ago. But I think what maybe is not, And I’m not sure if this is what you were alluding to, but I think what’s not super clear in open-endedness style search problems is do we frame all of them as basically like an unsolved, very difficult problem, or, like, where the objective is clear? If that’s not the case, I still don’t know yet entirely what the value of it is. maybe you have other opinions. Like, I don’t have too many opinions on this, but at least from my time at Sakana, like, I got the sense that, like, we ultimately still kind of wanna approach things the way that, like, say, OpenAI approached Navier–Stokes. We want. We-- There’s still a lot of nudging in certain directions that we want to have to, like, get to the point where we have something interesting.

Swyx [01:09:29]: My version of it is, like, maybe it’s a split between basic science and applied science. Basic science, you’re researching for researching’s sake.

Swyx [01:09:36]: You just wanna understand things better. I have no idea if, like, there will be any application at all whatsoever, but, that, And then applied, you have a goal.

Swyx [01:09:45]: You’re, you’re trying to minimize loss in some way? and so, what I really, think, in terms of, like, the big bets that people have, what if there was no prompt? Like.

Alex Zhang [01:09:57]: I see.

Vibhu [01:09:58]: You just pick domain and let it

Swyx [01:09:59]: Like, you just, like, you just spawned in this, like, swarm of things and you’re like, “Hey, what’s up, guys? Like, what you guys working on?”

Alex Zhang [01:10:03]: Yeah.

Swyx [01:10:03]: And, like, you just decide

Alex Zhang [01:10:05]: I see

Swyx [01:10:05]: Like, this is an interesting problem.

Alex Zhang [01:10:06]: The biggest issue that, like. And maybe there’s a clear path to this, but when I was there at least, the biggest issue that people had with open-endedness was like, how do you ultimately choose at the end? How do you pick out the interesting stuff? Because when there is no. Like, maybe the agent comes up with a goal, but in a lot of cases, like, what they. And they have something called Fugu, I think, which is like a It’s like a model router type thing that was, like, inspired at least by this idea of, like, let’s pick a problem where maybe we can pick out the best solutions to something. In this case, it’s like pick the best model for this problem.

Swyx [01:10:41]: You’re the first person to connect model routing to open-endedness.

Alex Zhang [01:10:43]: No, yeah. But, so I bring this up because I think, like, with open-endedness, like, just generally the issue is, like, when we have this giant corpus of, like, slop, like, how do we sift through

Swyx [01:10:57]: Yes

Alex Zhang [01:10:57]: And find, like, the hidden gems? And, like, the solution to open-endedness really just letting models run forever and, like, finding. Like, just doing data gen-- just doing super high throughput data generation and then, like, asking agents to go through and, like, find meaningful things. Like, I’m not sure. Maybe that’s sufficient? Like, that would be, that would be cool.

Swyx [01:11:21]: To me, it’s, like, very interesting as a counter to basically all of machine learning Where you have a goal, to have no goal.

Alex Zhang [01:11:28]: Yeah. Yeah.

Swyx [01:11:29]: But, or, like, an ill-defined goal that you’re like, “Well, what about this goal?” And you’re like, “Well, okay, maybe.” And then you, like, sort of research more and you find

Alex Zhang [01:11:36]: Yeah

Swyx [01:11:36]: That is an interesting goal. ‘Cause, like, I think, like, finding the objective function, like you said, like, Jeff found an objective function That was interesting that no one was exploring.

Alex Zhang [01:11:43]: Yep. Yeah.

Swyx [01:11:44]: I think that is, like, similar to your message about grad students as well. Like, you stay in school because you are. you want to pursue open-endedness. If you want to, profit max and, like, join the, escape the permanent underclass, then you join a lab.

Swyx [01:11:59]: Right?

Alex Zhang [01:11:59]: Yeah. Yeah. It’s funny. I feel like I don’t, I don’t hear this discourse a lot. I’m, I’m in the East Coast, so it’s, like, a very different type of. But then when I. whenever I come here, it’s like, that’s always, like, the topic of discussion.

Swyx [01:12:12]: You cannot pay rent without doing this.

Alex Zhang [01:12:13]: Yeah.

Swyx [01:12:15]: Yeah, you’re getting priced out, guys.

Vibhu [01:12:16]: Yeah.

Swyx [01:12:17]: Okay. So, yeah, there’s, there’s all that. I don’t know if you wanna-- if it’s relevant, enough to talk about the mismanaged geniuses, which you were pulling up.

Vibhu [01:12:25]: No, it’s just on your blog. But I will poke on, Sakana.

Swyx [01:12:28]: Oh, Sakana? Oh, okay.

Vibhu [01:12:28]: Yeah. So they did in their, blog post, I talked to them about this as well. So one of the cool results of Ultra is they basically just let it loose on automated data science research

Sakana AI and Weird Research Bets

Vibhu [01:12:40]: With little to no human intervention. It’s just kind of making

Swyx [01:12:44]: Yeah, so this is auto research, which is a little bit more open-ended, and there’s degrees of open-endedness, and I agree with that.

Vibhu [01:12:51]: Yeah, separate than auto research with objective, this is just

Swyx [01:12:54]: Yeah

Vibhu [01:12:54]: Do stuff. But, it’s cool. They’re, they’re working on it for those that are interested.

Swyx [01:12:58]: While you’re bringing it up, actually, what is your take on Sakana? Like, what are they doing apart from being, the Japan one?

Alex Zhang [01:13:04]: Yeah.

Alex Zhang [01:13:05]: I actually love the people there. Like, I think they have a really smart team. and it makes sense. it branched off from, like, an earlier team at GDM, which was also kind of, I guess, doing this kind of, like, open-ended evolutionary research style stuff. What I liked about my experience there, at least, was that they did have that, like, mishap back, I forget, at this point when, but I think, like

Swyx [01:13:31]: You’re talking about AI scientists?

Alex Zhang [01:13:32]: No, the GPU kernel.

Swyx [01:13:34]: Oh, okay, yes.

Alex Zhang [01:13:35]: That one. Yeah.

Swyx [01:13:36]: People cannot forgive them for that. Yes.

Alex Zhang [01:13:37]: Yeah. And I guess, like, the AI scientists, like, there’s, there’s some criticisms of it that I don’t, I don’t work on that, so I have no kind of take on it. But I think in general, like, what I like about them at least is that they’re a little bit more of a researchy type lab. So, like, they don’t operate in the same space as, like, OpenAI or Anthropic. Like, for sure, like, definitely no. They do not. it’s pretty obvious probably that, at least when I was there, they do not have a big competitor model or something that, like, that everyone is using. But I think they kind of operate in some ways as, like, a PhD lab, which is cool. Like, and I think, like, David Ha is, like, he’s, he’s really smart. Like, I think he has a good sense of, like. Also, I think the market in Japan is also a little bit different for AI, and, like, who they’re targeting is slightly different than maybe what we’re used to here. But yeah, I like that they take kind of. A lot of their research is kind of weird, I think, when people view it? And I like that. Like, I think it’s

Swyx [01:14:34]: We should have more weirdness, yes.

Alex Zhang [01:14:35]: Exactly.

Swyx [01:14:36]: And you said different market. Just, is it, like, enterprise?

Alex Zhang [01:14:39]: Like, the way it works there is a bit different, like how deals happen and stuff like that.

Vibhu [01:14:44]: They do have a. I guess this page is originally in Japanese, but they do have a model

Alex Zhang [01:14:49]: Oh

Vibhu [01:14:49]: Specialized for the Japanese market.

Swyx [01:14:51]: So you didn’t know that.

Alex Zhang [01:14:51]: I didn’t know that.

Vibhu [01:14:52]: I didn’t know it too.

Alex Zhang [01:14:53]: Did not know this.

Vibhu [01:14:53]: I also have personal friends that know the team.

Alex Zhang [01:14:56]: Yeah.

Vibhu [01:14:56]: So there is a. Even from the sense of a way that you speak culturally Responses are tuned towards that. This is not like it’s frontier on benchmarks. It is a cultural appropriate model for them, and then they have, like, chat and all that.

Alex Zhang [01:15:11]: Yeah.

Vibhu [01:15:12]: But, to mirror your point, there’s also, like, how should education look like? And someone wants to work on it, and they’re a very PhD lab of, “Do your thing. Why not? We have money. Go research.”

Swyx [01:15:22]: Oh, they say it’s a Kimi fine-tune. That’s nice.

Vibhu [01:15:24]: Oh, there you go. Kimi.

Swyx [01:15:25]: Good for them.

Swyx [01:15:26]: Yeah, and, speaking of Kimi, right, like another, just a grad student that spit out and like, yeah, I just have this, like, Kimi delta attention that wants-- that I wanna work on.

Vibhu [01:15:36]: Yeah. Yeah.

Swyx [01:15:36]: And, like, somehow managed to make Moonshot. Don’t understand it still.

Alex Zhang [01:15:41]: Yeah. Well, he’s, he’s super cracked, at least my understanding. I think in general, like, a lot of, a lot of the Chinese labs have done really cool work.

Kimi Swarms, Dynamic Workflows, and Convergence

Swyx [01:15:51]: Yeah.

Alex Zhang [01:15:51]: Like, yeah.

Vibhu [01:15:52]: Any thoughts on Kimi agent swarms?

Alex Zhang [01:15:55]: Yes. one thing I will say is whatever OpenAI is doing with their agent swarm is, like, clearly the right thing to do. You have to kind of think about it this way. Like, nothing, especially without, like, a very smart harness design, which I don’t, I don’t think anyone really has so far, Nothing comes so easily for free. For example, like, the agent swarm design is not something you can just take for granted. Like, it’s not like GPT-6 Astra is just super smart and then it just got agent swarms running well. They clearly trained. the Hugging Face incident was them training a system to be like a swarm. and I think, like, clearly they’ve done something really well to the point where you can throw 40 million dollars and solve an unsolved problem. And I think with the Kimi agent swarm thing, like, at least from when I read it just came off as like, this is interesting, but I don’t actually know whether or not this can solve anything novel.

Swyx [01:16:59]: Yeah, they just. They were like, “It makes spreadsheets for you.”

Alex Zhang [01:17:01]: They kind of were just like, yeah, like, here is a, here is a swarm that, like, kind of does stuff, and it’s cool. but. And I will say the same thing with dynamic workflows. I actually kind of think the dynamic workflows release was sort of a flop. I don’t know how you guys feel about it, but I think it’s like. my understanding is, like, it’s not used that often or, like

Swyx [01:17:21]: It’s just very expensive.

Alex Zhang [01:17:22]: It’s too expensive and like

Swyx [01:17:22]: It’s ultra code. It’s basically like, take over my bed.

Alex Zhang [01:17:26]: And it doesn’t I’ve tried it, and, like, it doesn’t act in the way that, like. Again, I’m not, I’m not the biggest OpenAI, like, stan or something, but I think whatever they did was very impressive. Like, they somehow managed to get a way for this swarm to actually act

Swyx [01:17:41]: I see

Alex Zhang [01:17:42]: Towards a goal. And yeah.

Swyx [01:17:44]: I see.

Alex Zhang [01:17:44]: It’s very difficult.

Swyx [01:17:45]: I see. So, like, efficiency of the multi-agent swarm is the objective function here.

Alex Zhang [01:17:51]: Yeah.

Swyx [01:17:51]: Right?

Alex Zhang [01:17:51]: Yeah.

Swyx [01:17:52]: Like, how much of this is slop? Like, this is a lot of slop.

Alex Zhang [01:17:55]: Yeah, I think so.

Swyx [01:17:55]: OpenAI is less slop.

Alex Zhang [01:17:56]: We take for granted what it means for a swarm to converge to an answer.

Swyx [01:18:00]: Yeah.

Alex Zhang [01:18:00]: It’s just like. It’s not something we take for granted.

Swyx [01:18:02]: Yeah. Yeah. We’ve, we’ve done one pod with Noam Brown and, like, his

Alex Zhang [01:18:06]: Oh, yes, I remember. Yeah

Swyx [01:18:06]: His thing. His whole thing was like, okay, like, we’ve worked on a lot of, like, competitive agents. we’re working on collaborative agents.

Alex Zhang [01:18:13]: Yeah.

Swyx [01:18:13]: And, like, that’s now called a swarm.

Gemini, GDM, and Harness Engineering at Scale

Alex Zhang [01:18:15]: Yeah.

Vibhu [01:18:15]: I would do a quick poke. Do you have any thoughts on Gemini? They were also IMO gold. Like, there was a time where they were getting agents to reason for a long time and.

Alex Zhang [01:18:26]: Yeah,

Vibhu [01:18:28]: Is it too old to think about?

Alex Zhang [01:18:29]: No. I think it’s a little bit blown out of proportion. Like, Gemini. For, again, I, so I should preface by saying I haven’t worked at any of these places, so take this with a grain of salt, right?

Vibhu [01:18:39]: You have strong opinions on agent harnesses

Alex Zhang [01:18:42]: Yeah

Vibhu [01:18:42]: And, this was

Alex Zhang [01:18:43]: But, let me just say this first. This work was really impressive. I think what they showed here was, like, they took a time when the models weren’t that good

Vibhu [01:18:53]: Yes

Alex Zhang [01:18:53]: And they managed to be very smart about, like, what the harness does. I remember for this at least, like, yeah, like AlphaGeometry, I guess that was a year before this, but it was very cool. They took it to the max, and they, like, designed, I don’t know. I’d say I’m not too big on competitive math, but I think, like, GDM, it’s sort of a shame. Like, everyone I’ve talked to about GDM kind of has the same opinion, which is that it’s way too, like, bureaucratic. Whatever is they have the talent and the resources to do almost anything, but, like, I don’t know, until they figure that part out, like. Nothing against anti-gravity, for example, but, like, I don’t know anybody that uses anti-gravity. And so I’ve tried it once, and it’s. I don’t see a reason to switch to it. and I think for whatever reason, like, they’ve been struggling with this, so, yeah.

Swyx [01:19:40]: Yeah. Well, a lot of people dogged on Meta for a long time until they started

Alex Zhang [01:19:44]: Yeah, and they recovered

Swyx [01:19:44]: Coming out. And, like, I think, Google’s going through that phase right now.

Alex Zhang [01:19:47]: Yeah.

Swyx [01:19:48]: And, it’s, it’s just you gotta stay alive and

Alex Zhang [01:19:51]: Yeah.

Swyx [01:19:52]: I wanna focus back on

Alex Zhang [01:19:53]: Yeah

Swyx [01:19:53]: Just, like, your thoughts, just general. we can talk about speculative PTC

Speculative PTC and Parallel Tool Execution

Swyx [01:19:58]: Mismanaged geniuses, or just, like, throw away all this and just talk about whatever else.

Alex Zhang [01:20:03]: Okay, let’s, let’s talk about mismanaged genius for a little bit.

Swyx [01:20:06]: Yeah.

Alex Zhang [01:20:06]: I, the only comment I’ll say on speculative PTC is that it’s a really simple idea. It’s almost, like, obvious that this should be done, and, like, there’s not much more to talk about it. Like, I think it’s just, like, you should just use it for, like, coding. Like, anything with programmatic agent calling, like RLMs or Kodak, like, yeah, it’s like, it’s like a no-brainer.

Vibhu [01:20:25]: What’s the, for people that haven’t read it

Alex Zhang [01:20:27]: Yeah

Vibhu [01:20:27]: What’s the one-liner for people?

Alex Zhang [01:20:29]: The simple thing is when the model is, writing its code or, like, even as, like, after it finishes writing the code, a lot of tools tend to be, like, sequential or, like, you have to wait on them, so you should just launch them in advance. Like, if you’re able to jit compile this code, you can probably figure out, like, even though it’s, like, kind of variables and stuff, like, you can figure out, like

Swyx [01:20:52]: Yeah, statically analyze.

Alex Zhang [01:20:53]: Yeah. So

Vibhu [01:20:54]: Speculation.

Alex Zhang [01:20:55]: Yeah. There is this, Someone pointed to me some actually, like, academics have, especially PL, like programming languages people, have some, like, very kind of cool ways of doing this. And so, like, at some point maybe I’ll, I’ll, I’ll, like, work on this.

Swyx [01:21:09]: I guess mostly you have to change language, because if you are in JavaScript, Python, you can’t do this.

Alex Zhang [01:21:13]: Yes. Yeah.

Swyx [01:21:14]: So, like, Haskell, yes. what’s, what’s the, what’s the normal one that’s, that’s not Haskell?

Alex Zhang [01:21:21]: Lisp.

Swyx [01:21:22]: Lisp, OCaml.

Alex Zhang [01:21:24]: OCaml, oh, yeah.

Swyx [01:21:25]: Yeah, any functional language

Alex Zhang [01:21:25]: Yeah

Swyx [01:21:25]: You can actually, like, pipeline this.

Alex Zhang [01:21:27]: Yeah.

Swyx [01:21:27]: So Effect-TS if you wanna do TypeScript.

Alex Zhang [01:21:29]: Yeah.

Swyx [01:21:30]: Okay, we can switch over to,

Vibhu [01:21:32]: I like this diagram.

Alex Zhang [01:21:33]: Yeah.

Swyx [01:21:34]: Which is like your, you guys’ whole thesis, right?

Capability Overhang: Reliable Long-Running Work

Swyx [01:21:36]: Like, that, to me this is, like, kind of like a restatement, but maybe I’re missing something of

Alex Zhang [01:21:41]: Yeah

Swyx [01:21:41]: Like, well, work on better harnesses or, like, your models actually are capable a lot more if you try harder, so this is a skill issue.

Alex Zhang [01:21:48]: Yeah. Yeah, basically. I think there’s one thing I want to see. I appreciate that there’s a big focus on, like, jagged intelligence, because it paints a big picture of, like, we can do this if we really set our minds on it. But I kind of wish. And maybe someone in academia should do this. Like, really just sit down and think about, like, if I took Astra, even the current frontier models are not good enough at, like, doing a particular job over, let’s say, the span of a month consistently and well. And I think this is, like, a stupid problem. Like, I genuinely think we can solve this. You don’t need to be a frontier lab and, like, do all this, like, fancy stuff for your IPO. Like, I think, like, these models are so smart that even if it’s, like, a silly way, I think that it genuinely is a skill issue of you can get a model to be as good as, let’s say, like, just some 18-year-old high school kid

Alex Zhang [01:22:50]: Doing some job. I think it’s, like, ridiculous that we can’t do that. And it’s. Part of the reason is, like, the format of a language model is not really amenable to that, but I think you can shape a harness around it and do it. And I think, like, this in itself is, like, ignoring the RLM stuff, ignoring all the, like, what abstractions should we use? Like, I just think someone can design a harness that can do this. Like, I, that’s. Maybe it’s

Swyx [01:23:13]: When you say do this, do what?

Alex Zhang [01:23:15]: Do long-running but simple tasks, and do them reliably.

Swyx [01:23:20]: Okay.

Vibhu [01:23:20]: What’s an example? So is this different than, like, pick your favorite company, Harvey, for example Using LLMs to do legal work, or what’s the.

Alex Zhang [01:23:30]: I guess it’s kind of like that, except if the bottleneck was not, like, certain legal knowledge or something. Like, I don’t know. Let’s say,

Vibhu [01:23:38]: I guess, like, the examples, people can take models and build pipelines or whatever and Have an agent repeatedly do whatever it has they want, right?

Alex Zhang [01:23:47]: Yeah. So for example, like, if I wanted a general system that I could kind of talk. I can talk to it like I would talk to an intern and basically just ask it to do. to explore some small thing. So maybe an example of this is, like, very silly auto research is maybe an example of this, of, like, not necessarily finding super novel solutions, but at least optimizing all of the easy parts of any problem. They often end up being over-indexed for, like, ML training and things like that. But yeah, I don’t know. maybe that’s, that’s, that’s not, like, super clear, but there is a lot of People’s general workflows where you probably could just vibe code up. Some specific harness to help you do, like automate this thing. Some examples are like automating, finding, like research papers and stuff like that. But usually people will design like a specialized agent to help them do this kind of thing. Or like they’ll vibe put a harness, like, and just run it or like their Slack bot or something. But I almost think there’s just like a standard form, like just a harness that you just plug in. Like you don’t-- It doesn’t need to be designed for finding papers or fi-- Like, you just kind of tell it, find this for me, and you like plug it into that setting. What I’m getting at is that I think there’s a lot of easy things that can be automated. And

Vibhu [01:25:13]: Is this like a hypothesis or a point around like capability overhang? Like even if we paused, there’s still a lot of impact to be had with current state of models?

Alex Zhang [01:25:22]: In some sense, yes. Like, I guess what I’m presenting is the easiest form of this. But what this is kind of saying is that, like, we have jagged intelligence on a lot of things. Like, for example, models are like disproportionately good at coding and math. This is saying like we can translate those abilities to many other things. So like for example, if you took-- Usually if you take someone who was an IMO gold or something and you kind of apply them to a lot of different problem-solving domains, they can figure it out. I don’t actually know if this applies to models. For example, like in GPU code optimization, one very interesting question is whe-- if you were to take out all of the GPU programming data, like from a model, but it was re-- it was like as good as Astra is now, just without, like with that taken out, would it be able to still optimize GPU kernels? like would it be able to learn in context roughly what it needs to learn and then like have some pipeline or like come up with some solution to solving like optimization tasks? And I think like there’s like a mismatch between, like if you took a human that was as smart or like knew as much as Astra, there is a mismatch between what that human can do and what Astra can do, maybe around a harness. and I think we can actually approximate the human a lot more.

Continual Learning and General Problem Solving

Swyx [01:26:42]: To me, it sounds very approximate to the continual learning problem. I think you’re

Alex Zhang [01:26:46]: That’s the best example. Yes.

Alex Zhang [01:26:48]: Yes.

Swyx [01:26:48]: Why didn’t you just say that then?

Alex Zhang [01:26:49]: Yeah. I guess I

Vibhu [01:26:50]: I was like, I was like thinking, could I just blurt out some words like learning?

Alex Zhang [01:26:54]: I’m careful with that. But yes, I

Vibhu [01:26:57]: You have a very eye.

Alex Zhang [01:26:58]: Maybe, Maybe, like a bit of a, I don’t know, like a

Swyx [01:27:03]: No. So I think you’re a very, yeah, I don’t know, I don’t know your undergrad actually. Are you like a math person generally, or

Alex Zhang [01:27:09]: A little, yeah. Yeah.

Vibhu [01:27:09]: Did you study math?

Alex Zhang [01:27:11]: Yeah, that’s what I wanted to do at least.

Swyx [01:27:12]: Like a category theory type of, abstraction where you think in categories and then you have to like then translate down to the specific. And, but then you like actually really care more about the category.

Swyx [01:27:24]: And like that’s the communication error because like everyone’s listening for the specific, but actually trying to also, convey the general.

Alex Zhang [01:27:31]: Yeah.

Swyx [01:27:32]: Which is hard. I don’t really know. you can, maybe use like a shorthand of like, “Okay, I’m at level two and then I’m gonna go up to level three Then come back to level two.” That we should have say some like epistemic, like shorthand for like this kind of thing.

Alex Zhang [01:27:45]: Yeah.

Swyx [01:27:45]: Because it’s hard. Like you’re, you’re compressing a lot into word after, like sequential word decoding.

Swyx [01:27:51]: Should we convert to Neuralese? is there like a, a better form of output than English or, Python or JavaScript? I don’t know.

Neuralese, Programming Languages, and Diffusion Thinking

Alex Zhang [01:28:06]: Yeah.

Swyx [01:28:06]: This is very kind of like a shit post, but like people have speculated about like what is the native language that people want to out-- that models want to output?

Swyx [01:28:14]: Some people say binary. That’s, that’s Marc Andreessen’s thing. I don’t know. Yeah.

Swyx [01:28:19]: PTX?

Alex Zhang [01:28:20]: Let’s say a mix of English and Python. And I only say this because The capability of a model is somewhat a reflection of what we train them on. So we still want like. Yeah, I don’t really buy the binary argument. I guess I, like, I understand, but it’s like

Swyx [01:28:40]: Yeah. You wanna model the world in some way. I think the,

Alex Zhang [01:28:43]: Yeah.

Swyx [01:28:43]: One thing I’ll, I’ll bring up is always, which I always do in this kind of conversation is Sapir–Whorf Which is you, if you choose English, you will have locked into however long English has been around, which is, let’s say five hundred years, which is not that long. Like actually Like what you, the language that you speak constrains how you think.

Swyx [01:28:59]: And if you learn a different language, for example, someone, in Chinese, we don’t have tenses. I don’t know if you. I actually didn’t know that. And I speak Chinese.

Alex Zhang [01:29:09]: Oh. I did know that, but my Chinese is not great.

Swyx [01:29:12]: Okay.

Alex Zhang [01:29:13]: Yeah.

Swyx [01:29:13]: Yeah. Or like, in, let’s say in Japanese or, I know, I forget what language it is. Like in Korean, everyone you speak to, yeah, you have to like acknowledge social status.

Alex Zhang [01:29:23]: Yeah.

Swyx [01:29:24]: But it’s a different dimension than gender, right? Like, it just like, it just influences everything you do. when I take Ling 101, apparently there’s a, there’s a, there’s a language in Africa where like there’s a vegetable gender. yeah, right? Like just like you have. Or like Eskimos know no word for snow or whatever. Like, anyway, so like the language that you adopt affects your thinking. And if you Choose to output your chain of thought in English, you are biasing towards whatever English solves. I don’t know what the sort of prior of English is.

Alex Zhang [01:29:51]: That’s interesting. I did not think of it that way.

Vibhu [01:29:55]: At some level it’s interesting, right? So you’re right on language. a lot of model chain of thought also fluctuates language.

Vibhu [01:30:03]: The obvious example is Chinese models Speaking in English might still reason in Chinese. but at the same level, most models are very capable multilingually.

Vibhu [01:30:13]: And that adaptation we can see, you can add in languages. You don’t get that much from adding a whole language, but

Swyx [01:30:18]: Yeah.

Vibhu [01:30:18]: They will reason interchanged

Swyx [01:30:21]: Yeah. So we’re all autoregressive. but also like

Vibhu [01:30:23]: Right.

Swyx [01:30:23]: Let’s say German, like, subject-object, agreement, you have to put the verb at the end, which is very super annoying, like very famously. yeah, right. You don’t know what you’re doing until the end where you’re like, “Oh, that Mess of nouns and then the verb.” Well, the most classic one that most people be-- have heard of is Arrival, where, they have the heptapods where they think, that time is like flat to them. So they think in, they output entire sentences at one shot. so it’s, this is closest to, like, the difference between autoregression and diffusion.

Swyx [01:30:55]: We talk in autoregression. What if you could talk in diffusion Where things just resolve over time?

Alex Zhang [01:31:01]: I see. I see.

Swyx [01:31:02]: But like the whole idea shows up at once.

Alex Zhang [01:31:05]: I see.

Swyx [01:31:07]: So that is a drastically differenting, language, but it is a language.

Alex Zhang [01:31:10]: I see. Oh, that’s really interesting. That’s really interesting.

Swyx [01:31:12]: Which, like, machines could speak, that we, probably will never speak, but like, yeah, machines don’t care.

Alex Zhang [01:31:19]: Maybe this is a huge tangent

Swyx [01:31:20]: Yeah

Alex Zhang [01:31:20]: But are there not, like, things inherently that are reasoning chains that are inherently autoregressive?

Swyx [01:31:30]: Yeah, time.

Alex Zhang [01:31:30]: Sometimes, like. Yeah. Or like, yeah.

Swyx [01:31:32]: Yeah. Something happens first, then something else happens.

Alex Zhang [01:31:34]: Even like, yeah, anything in code, for example, like has to be causal in some-- usually at least has to be causal.

Swyx [01:31:40]: Well, no. so it’s a. Then you have to. Then you’re not exploring enough,

Alex Zhang [01:31:44]: That’s true. Yeah

Swyx [01:31:45]: Programming language theory, where, everything is like pure functional and like completely relational And, you sort of abstract away the solver that translates the relationships that is, are always true into code. So I, yeah, I feel like this is maybe a little bit too out of my depth.

What Comes After RLMs?

Alex Zhang [01:32:01]: No, it’s interesting though. Yeah.

Swyx [01:32:01]: But I love languages Whether it’s coding or human, and I do think a lot about how that affects reasoning and the boundaries of what we can do. I don’t need to go too much beyond that. I don’t know if you have any other thoughts. my closing question was gonna be, you have all these research, directions that you wanna do. You’re, you had a GPU mode phase. You had a, RLMs phase.

Swyx [01:32:25]: Presumably you have other stuff planned, which is why you’re not, doubling down on that. By the way, I notice that it is interesting how you guys do start with the GPU side, and then you migrate towards the zero gradient side, it’s, which is what Shenyou called it. Doesn’t that feel less legit than messing with GPUs?

Alex Zhang [01:32:44]: Yeah, I guess in the sense that, like. So you did bring up that, like, I like to think about things in, like, a math-oriented way.

Swyx [01:32:51]: Category, yeah.

Alex Zhang [01:32:52]: And it’s, like, very uncomfortable sometimes to be working on, like, harnesses and agents because it’s so.

Swyx [01:32:57]: Because you think all harnesses are the same.

Alex Zhang [01:32:58]: Yeah. So it’s like super fuzzy.

Swyx [01:33:00]: So like, you just, like, two new ideas in harnesses. Got it.

Alex Zhang [01:33:01]: Yeah. It’s, it’s It’s, it’s also just, like, empirically it’s hard to, like, verify a lot of findings, at least with the compute that we have available to us. But the reason why I think I’ve moved on to a lot of these problems is I think actually this is where most of, like, the innovation is yet to happen. To me, like, the GPU level is a means to exploring other ideas. Like, you want to, for example, like, get good at writing kernels or, like, even automate writing kernels for the sake of a broader goal of, like, I want to explore ideas where I’m not bottlenecked by systems challenges. In that sense, like, I guess a lot of what’s written there is all harness stuff, but I am also interested in things at the model level as well. but I’ll just leave it at that.

Swyx [01:33:48]: Okay.

Alex Zhang [01:33:48]: Yeah.

Swyx [01:33:48]: That’s a good hint. anything, any. If people wanna reach out to you, what are you looking for help on? What do you want collaborators on? any sort of calls to action?

Collaborating on Research and Choosing Big Bets

Alex Zhang [01:33:58]: Yeah. So, I guess there’s nothing I have in particular where I feel like I need to work with someone on, unless it’s, like. unless it’s with a company before, like, compute or, like, with. to talk with other people about it. But I will say I’m not. I’m never opposed to working on ideas with other people. I get reached out to a lot by often undergrads or even, like, other students.

Swyx [01:34:22]: Podcasters.

Alex Zhang [01:34:23]: Podcasters. and usually I feel like I get an email that’s something along the lines of, like, “I really like RLMs.” Like, “I wanna work together.” and I feel like I

Swyx [01:34:33]: Yeah, that’s a bad reach out, right?

Alex Zhang [01:34:35]: Yes, yeah.

Swyx [01:34:35]: The worst is like, “Can I pick your brain?”

Alex Zhang [01:34:36]: Yeah.

Swyx [01:34:37]: And like, “On what?” Like, “Read my paper, dude.” Like.

Alex Zhang [01:34:39]: They’ll like, they’ll be like, “I read your paper,” in quotes, like, “Recursive language models,” or like, “Prime Agent,” like a self-improving RLM harness or something. and it’s kinda like I like. I really like people that are opinionated, even if we disagree. I think if you have strong opinions and are able to, like, think through why you think those opinions are right or wrong, ‘cause usually it’s, it’s hard to actually tell. But, like, you have strong convictions about certain problems. Like, I’m, I’m always happy to, like, chat and even, like, potentially work on something together. I have, like, no limit to who or, like, what I would like to work on. So, yeah.

Swyx [01:35:12]: No limit?

Alex Zhang [01:35:13]: Yeah. I. in the era of agents, I think there’s a lot more work you can do, like, bandwidth-wise. So I. Yeah. I think in general, like, I am not hard to impress, but I think it just takes a little bit of effort to

Swyx [01:35:30]: Yeah

Alex Zhang [01:35:30]: Kind of. Yeah, know

Swyx [01:35:32]: Yeah

Alex Zhang [01:35:32]: Know what you want.

Swyx [01:35:32]: It’s very clear. and like, when you see a new thing come out, well executed, good, simple idea, then, like, get that

Alex Zhang [01:35:40]: Yeah

Swyx [01:35:40]: Immediately gets your attention, right?

Alex Zhang [01:35:41]: I get excited. Yeah.

Swyx [01:35:42]: It’s actually, like, not that hard to get the same attention that all the Frontier Lab guys

Alex Zhang [01:35:45]: Yeah

Swyx [01:35:45]: Because they are looking for you. you just have to put yourself out there, right?

Alex Zhang [01:35:48]: Exactly, yeah.

Swyx [01:35:49]: But yeah, it’s true. I will say, I think human attention very scarce right now, and I do struggle with, like, the number of projects I have going on.

Swyx [01:35:57]: And, I don’t know how to manage it. I don’t think agents are helping at all.

Swyx [01:36:00]: Like, I will just prompt it and. I’ll prompt a thing and then never look at it.

Swyx [01:36:03]: Right? Like, which is very common.

Alex Zhang [01:36:05]: Yeah.

Swyx [01:36:05]: Yeah, and that sucks.

Alex Zhang [01:36:06]: I guess, maybe the. one of the smaller differences in, like. actually, maybe you were doing research. I’m not sure. But for me at least, like, I’ll have maybe, like, 10 or 15 different ideas that I wanna do, but the thing is, like, most of them are bad.

Swyx [01:36:20]: Yeah.

Alex Zhang [01:36:20]: And this also maybe is true even for someone that reaches out to me. Like, maybe the idea is actually bad, but it looks interesting to me. And so, like, we can spend, like, some time looking into it, and if, like, we feel like there’s actually something there, like, then we’ll. we should take the next few weeks and just really pursue it. And, like, this is my style with. This is why I love the PhD, by the way, because there are times when I’m just thinking about problems, like, maybe on a run or just, like, playing tennis or something. Like, I’m not, I’m not working, I guess. But it’s like those are the most fun times, and then when I, like, really am convicted about something, I’ll just, like, drop everything and just do it.

Alex Zhang [01:36:52]: Like, just spend, like, all my time thinking and working on that problem. And then, once you get to the point where, like, you can just run experiments, then it’s, it’s, it’s kind of easy coasting again, so.

Science as the Next Frontier

Swyx [01:37:03]: Yeah, sorry. This is

Alex Zhang [01:37:04]: Yeah. No problem

Swyx [01:37:04]: Like the, for the fourth last question, which is like, I think a lot of people are also thinking about science as the next frontier, like physical sciences Bio, math even. how do you separate, like, I guess, let’s say your choice of projects that is applicable for industry And then maybe it’s part-- choice project is just, like, science?

Alex Zhang [01:37:27]: I actually worked on, like, AI for bio stuff before I started my PhD. The field has changed a lot since then

Swyx [01:37:33]: Yeah

Alex Zhang [01:37:34]: I should say.

Swyx [01:37:34]: ‘Cause it’s, it’s like, it used to be a theoretical, like, of course, what do you mean? Like, I have one path and then

Alex Zhang [01:37:39]: Yeah

Swyx [01:37:39]: I chose that. But now a lot of people are crossing over.

Alex Zhang [01:37:41]: Yeah.

Swyx [01:37:41]: And like, so we have started a science pod to just Cover those things

Alex Zhang [01:37:44]: Oh, wow

Swyx [01:37:45]: Because a lot of engineers are like, “Well, actually there’s, Tractable problems there.”

Alex Zhang [01:37:50]: Yeah. I will preface by saying my understanding of a lot of these topics is probably pretty limited. But I think, like, if I find out either because someone reaches out or, like, I look at a problem and I’m like, “Hey, like, some design principles that we use or that we’re thinking about right now actually make a lot of sense for this problem,” I get excited about those as well. But I think it’s harder. I don’t know. I think with. I think science, especially like empirical or, like, applied science has very long, like, what is it called? Like, feedback loops or whatever.

Swyx [01:38:23]: Yeah, it converges to a robotics question.

Alex Zhang [01:38:25]: Yeah. to me, like, also this aspect of, like, what is worth spending and betting my time on now? Because, like, maybe I spend a lot of time on this problem and then, like, in six months, like, a different solution, kind of like maybe a new model comes out and it’s like, “Oh, it’s way better for this.” And so I do have to be careful. I, like, you have to be conscious about, like, where you think things might be going.

Swyx [01:38:47]: Yeah, exactly.

Alex Zhang [01:38:47]: So, yeah.

Swyx [01:38:48]: Publish cycle.

Alex Zhang [01:38:49]: Yeah. So

Vibhu [01:38:51]: Which then I can just tell ARC-AGI-3, it’s saturated. We did it.

Alex Zhang [01:38:54]: Yeah, ARC-AGI-3 got saturated in less than a year, so it’s kind of ? Like, it’s. I don’t know. Like, if you were a lab picking that problem, like, you’re probably kind of sad now ‘cause actually

Swyx [01:39:04]: Yeah. Well, so exactly. That’s why knowledge work, gaming, all these things are saturated.

Alex Zhang [01:39:08]: Yeah.

Swyx [01:39:08]: Now they’re actually the frontier is science. So

Alex Zhang [01:39:09]: Yeah.

Vibhu [01:39:10]: Knowledge work is saturated.

Swyx [01:39:12]: Yeah. GDP val is like 80 something, 90 something.

Swyx [01:39:16]: Like, there’s, there’s 90 to 100% that was obviously gonna get

Alex Zhang [01:39:20]: It’s gonna be really hard to tell

Swyx [01:39:21]: 10 years. But, like, well, the next low-hanging fruit is gonna be

Alex Zhang [01:39:25]: It makes sense

Swyx [01:39:25]: The other stuff.

Alex Zhang [01:39:26]: Yeah. Maybe I’ll think about that more. I actually, I haven’t given too much thought to

Swyx [01:39:30]: I’m just trying to guess your next direction, actually.

Alex Zhang [01:39:32]: No. I will say because I think especially at MIT, like, it’s, it’s, it-- Yeah, there’s a lot of really talented scientists there, like, in the natural sciences. And I think it’s, it’s a little bit, like, sacrilegious almost to be like, “I’m gonna figure out, like, your problem.” Like

Swyx [01:39:45]: No, that’s how, that’s how it’s done.

Vibhu [01:39:46]: It’s great.

Alex Zhang [01:39:46]: Oh, no, I know. Yeah.

Swyx [01:39:47]: So I interviewed Yitai who did the IMO thing. He’s just like

Alex Zhang [01:39:50]: Oh, yes. Oh, yeah.

Swyx [01:39:50]: “I’ve never, I’ve never been to IMO. I don’t even know what it is.” It’s a skill model, dude.

Vibhu [01:39:55]: Yeah.

Swyx [01:39:56]: Which is, like, very disrespectful, but like, whatever. But that’s why, yeah.

Vibhu [01:40:00]: Yeah. At some point, like, you have to respect, like, okay, the progress is being made.

Vibhu [01:40:05]: Like, number is getting output, right?

Alex Zhang [01:40:07]: True. Yeah, true.

Swyx [01:40:08]: Yeah. that is a very big lesson. It’s very interesting, like, ‘cause the mathematicians are responding this way to NARI systems right now.

Alex Zhang [01:40:13]: Right. Yeah.

Swyx [01:40:14]: Like, Terry Turnstow is like

Vibhu [01:40:15]: Turnstow

Swyx [01:40:16]: “No, like, let’s not, let’s not use AI.”

Alex Zhang [01:40:17]: Yeah.

Swyx [01:40:17]: I’m like, “Mm, I don’t know.”

Alex Zhang [01:40:19]: Well, yeah. I think that whole thing is kind of weird ‘cause I feel like, I feel like they would have had a stronger case if a lot of them didn’t work with OpenAI before, like, all this happened.

Swyx [01:40:30]: No, that’s ad hominem, and they’re really trying to stay away from that. Then so what, right?

Alex Zhang [01:40:34]: So what? Yeah.

Swyx [01:40:35]: Like, I don’t know. Like, so what? They got. they’ve, they’ve collaborated. I collaborate with people that I don’t

Alex Zhang [01:40:39]: I guess it’s true

Swyx [01:40:39]: I don’t agree with or

Closing: Research, Academia, and What Comes Next

Alex Zhang [01:40:40]: That’s true

Swyx [01:40:40]: Whatever.

Alex Zhang [01:40:41]: Yeah. That’s true.

Swyx [01:40:41]: Or, like, I did a thing and then now I regret that. I changed my mind. whatever.

Alex Zhang [01:40:45]: Yeah.

Swyx [01:40:45]: So I’ll defend their right to say that. But like, yeah, a lot of people are reasonably disagreeing with them.

Alex Zhang [01:40:50]: Yeah.

Swyx [01:40:51]: Okay, cool. thanks for your joining us. Congrats on, your success so far. I can’t believe you’re still not done with your PhD.

Alex Zhang [01:40:58]: Well, it’s year two?

Vibhu [01:41:00]: Yeah. Can’t believe we did this podcast without going through the RL paper.

Swyx [01:41:05]: He had a, he had a definition.

Alex Zhang [01:41:07]: I think the paper is more about, like, empirical results. Like, the actual idea is quite simple.

Vibhu [01:41:11]: Yeah. And you’ve talked about it many times.

Alex Zhang [01:41:13]: Yeah, at this point. I think there’s, there’s more interesting things to look over now, so.

Vibhu [01:41:17]: Cool.

Alex Zhang [01:41:17]: Yeah.

Vibhu [01:41:18]: Well, we’re excited to see what you do next.

Alex Zhang [01:41:19]: Thank you so much.