Modal CEO Erik Bernhardsson 谈 AI 算力为何不是矩形

热烈欢迎Basil Chatha，我们最新的联合主持人，来自 AI 工程领域的前向部署方向！我们将与他一起推出一档新节目，并借此机会在他们完成巨额 D 轮融资之际采访了 Modal 的 CEO Eric！

两年前，我们与 Erik Bernhardsson 交谈过，了解他对真正无服务器 AI 基础设施的愿景：开发者应该能够运行代码，而无需配置集群、预测负载或为闲置资源付费。CTO Akshat 随后加入我们，讨论了 Modal 的基础设施原语和技术栈，以提供这些原语和智能体体验。

过去 5 个月发生了很大变化。自 50 亿美元 C 轮融资以来，Modal 现在报告其年化收入运行率突破 10 亿美元，以 160 亿美元估值完成了 7.5 亿美元 D 轮融资，并已启动近 30 亿个沙箱。在这次对话中，Erik 与 Vibhu 一起解释了这一增长背后的基础设施和经济逻辑——以及为什么算力需求很少像过去购买算力时使用的合同那样。

在 YouTube 上观看完整对话 →

GPU 需求不是矩形的

典型的容量合同为客户提供固定数量的 GPU，为期固定。将容量对时间作图，你会得到一个矩形。真实工作负载的表现则不同：它们会在一天中增长、激增和下降。为峰值购买足够容量，你就会为闲置硬件付费；购买较少，你就有可能错失需求。

Erik 将 Modal 的押注描述为一种时长转换：将数千名客户的需求汇聚起来，在底层确保容量，并让每个客户在需要时使用所需资源。聚合需求越平滑，这个资源池就越有用。这是对我们与 SF Compute 的 Evan Conrad 对话的有益补充：Evan 专注于让容量可交易；Erik 专注于让容量以更小、弹性的切片可用。

另一侧存在财务风险。如果提供商以当前价格锁定容量，而 GPU 价格随后下跌，它可能会持有昂贵的供应。Erik 表示，他并不预计会立即崩盘，但认为在构建业务时这一可能性很重要。跳转到 GPU 价格讨论 →

差点没能活下来的沙箱

在 Erik 和联合创始人 Akshat Bubna 注意到语言模型可以生成代码——并且需要有一个地方来运行它之后，Modal 推出了沙箱。早期采用率相当有限，团队曾讨论关闭该产品。随后，编码智能体、后台智能体和强化学习创造了新的需求。Erik 表示，沙箱已增长到约占 Modal 收入的三分之一，这是他在发布时没有预料到的结果。

这种增长也改变了基础设施必须做的事情。一个智能体可能会创建环境、运行代码、检查结果，并多次重试。启动和恢复这些环境的延迟成为体验的一部分。在我们最近对 Akshat 的采访中，我们探讨了从开发者体验到智能体体验的这一转变；在这里，Erik 补充了产品和商业故事。跳转到沙箱讨论 →

快速启动是一项基础设施特性

只有当容量能足够快地到位并发挥作用时，弹性才成立。Modal 的无服务器 GPU 文章描述了其中的机制：可用 GPU 缓冲池、自定义文件系统、进程检查点和 GPU 内存快照。这些组件减少了让另一个模型副本上线所需的工作量。这是一个具体例子，说明基础设施决策如何体现在应用的响应速度和成本上。

我们还讨论了 Erik 从 Spotify 推荐系统到 Modal 的历程、公司如何购买算力、NVIDIA 的替代方案、前置部署工程师，以及为什么在 Modal 代码审查是可选的。跳转到工程文化 →

节目笔记

Modal —— Erik 创立的 AI 基础设施平台。
Erik 的文章 —— 他关于基础设施、市场和创办公司的文章。
Modal 的 D 轮融资公告 —— 公司最新公布的 growth 和 sandbox 里程碑。
Modal 如何实现真正的无服务器 GPU —— 弹性推理背后的创业和扩展技术。
Modal 的资源求解器 —— 它如何在不断变化的价格和可用性之间选择容量。
扩展到一百万个并发沙箱 —— 关于沙箱扩展的工程记录。
我们与 Erik 的第一次访谈 —— 最初的无服务器基础设施论点。
我们与 Akshat Bubna 的访谈 —— 面向智能体的基础设施和开发者体验。
可编程数据中心 —— Evan Conrad 谈 GPU 交易与实物交割。

时间戳

0:00 Modal 做什么
1:25 从 Spotify 到创办 Modal
4:19 在生成式 AI 热潮之前创业
7:43 第一批客户与融资
10:00 为推理和自动扩缩容而构建
13:21 Modal 从哪里获得 GPU 容量
15:39 租用与拥有基础设施
19:45 GPU 价格下跌的风险
24:18 AI 需求是无限的吗？
26:12 NVIDIA 的替代方案
28:25 Modal 能从 neocloud 学到什么
30:26 GPU 作为一种资产类别
31:55 Modal 的产品：函数和沙箱
36:03 支持更小训练工作负载的理由
41:21 智能体沙箱正在如何变化
46:48 Anthropic 与沙箱安全
49:29 用智能体测试安全性
52:51 智能体时代的开发者体验
57:45 Modal 的前置部署工程师做什么
59:55 Modal 的产品管理方法
1:03:29 可选的代码审查与工程文化
1:05:56 云、本地部署和自带云
1:09:25 容量、融资与增长
1:13:07 扩展团队
1:17:28 Erik 的推文：临时沙箱
1:18:31 矩形 GPU 合同的问题
1:21:20 推理提供商与模型构建者
1:23:25 模型下一步在哪里改进

文字记录

自动转录自修订版；姓名和措辞可能存在错误。

0:00 感谢你来参加，Eric Fern-Hartson，来自 Modal。你能稍微解释一下这项业务，以及你是怎么走到今天的吗？好的。我们构建的，最简单的说法是，我们构建 AI 基础设施。所以我们提供一个计算平台和一个存储平台，让很多客户，比如 Dacagon、Suno、Runway、Cognition 以及许多其他公司，能够构建 AI 应用。所以这在实践中意味着大量的推理、大量的沙箱，有时是训练和微调，有时是批处理作业和其他事情。所以我们基本上构建了一个软件 SDK，让客户能够非常快速地行动并构建这些。我们管理底层的所有容量，让部署、扩展和在生产中运行这些东西变得更容易。你大概两年前上过 Light and Space，从那以后发生了很多事。你当时有多大，任何你能分享的数字，以及相比之下，你现在有多大？我其实不记得我们当时在哪，但大概是至少 50 倍，收入、用户和用户都是。是的。我其实用它做一个我自己的副项目。不错。是的。就是把 2D 图像变成 3D 模型。是的。所以就是在这里托管几个模型。你觉得怎么样？有什么反馈吗？我很喜欢。不错。好。我是说，是的，这就是为什么我用它而不是任何竞争对手之类的。好。所以在你创办 Modal 之前，你在 Spotify，还有 better.com，我想。

1:30 没错。是的。是的。你在 Spotify 做什么？我其实在 Spotify 做了很多不同的事情，但我的主要贡献是我构建了一个音乐推荐系统。第一个版本是我毕业后在 2008 年、2009 年加入的，大概，然后在 Spotify 待了七年。大部分时间基本上是从零开始构建音乐推荐系统，并建立了一个小团队，我们发布了一堆功能，比如相关艺人、Discover Weekly 以及一堆其他东西。所以大量的矩阵分解，大量计算密集的东西。可以说，后来成为 Modal 起源的很多东西，就是构建 AI 应用并意识到基础设施真烂。一定有更好的办法来做这个。所以很多，你知道，Modal 的理由就是构建一个我在 Spotify 那些年里一直想要的工具。是的。所以 Modal 更多是你在 Spotify 学到的东西的延伸，而不是来自 Better.com。Better.com 算是一段小插曲。那有点像过山车，关于这家公司说来话长。我们单独聊过。但我在 Better 做了五年 CTO，把技术团队从，我想，你知道，两三个人发展到三四百人。所以我学到了很多关于管理、组建团队和招聘的东西，也学到了很多关于开发者生产力的东西。我思考了很多关于工具之类的事情。做了更多全栈的东西。但是的，之后我有点回到了我内心亲近的东西，也就是基础设施、AI 和机器学习。Better.com 又是做什么的？抵押贷款。好的。是的。你是怎么进入那个行业的？

3:00 我的意思是，我们不必在这上面花太多时间。对，不，没关系。比如这个叫 Zach Frankel 的人，我当时并不认识他，后来他把我招进 Better 做 CTO，当时公司只有 10 个人。你知道，不管什么原因，创始人 Vishal，非常有意思的人，非常非常聪明，有点疯狂，但主要是正面的那种。我们相处得非常好。所以我们，对，2015 年开始做这个，我在那家公司花了五年时间把它做大，然后，你知道，最终发现做抵押贷款挺难的，而且疫情之后利率涨得非常快，这给公司带来了很多问题。但到那时我已经离开了。你离开是在 2020、2021 年左右？对，差不多吧。对。所以，好，你是在有模型这个想法之前就离开了，还是离开之后才开始做的？很难说。我离开 Better 的时候有很多随机的想法，我其实不确定自己想不想创业。我当时有点，你知道，疫情期间，我几乎算是休了个长假，有两个孩子。我是说，我现在还有孩子，但我是说，当时他们大概一个两岁一个四岁，所以我决定就待着，休息了一段时间，有点半职地工作，捣鼓一些东西，也不确定也许我该做点咨询，然后我开始做东西。我就对这个最终变成 model 的想法超级兴奋。但其实两者都有，你知道，model 大概是我做的第 30 个，你知道，所以在那之前有几次尝试都没成。

4:30 但我的模型是这样的，最初的想法是，这挺酷的。我应该做这个。这可能会是一个很酷的平台。其他想法是不是也类似，比如开发者工具？第一个想法是某种网页抓取的东西，但没什么进展。不过实际上，事后看来，也许本来能有所发展。这是在 ChatGPT 发布之前吗？是的。是的。2021 年。哦，好的。所以 ChatGPT 是 2022 年底发布的，对吧？然后我开始构建，所以也许稍微回溯一下背景，当我在 Spotify 工作时，我发布了一个开源框架 Kaluigi，它是工作流编排。所以算是同一类，Airflow 和 Prefect 之类的，某种程度上算是过时的类别，人们不太常谈论了。但有一段时间，它获得了不少关注。总之，我从未把它变成生意。但几年后，在 2021 年，当我在思考商业想法或要做的事情时，我有点回到了工作流编排。我开始捣鼓一堆想法，比如在云端运行代码，基本的编排代码在云端运行。我意识到的是，归根结底，核心难题不只是编排计算机，而是执行计算机。我想，这实际上是个更大的问题要解决。所以某种程度上，我实际上把它搁置了，然后说，实际上，我要去做这边这个其他事情，我会回来的。我会回到工作流调度上。但是，长话短说，我再也没回去，最终构建了 Modal，它的重点更像是，让我们想办法让在云端计算机上运行代码变得容易。最初的重点是相当通用的。

6:00 实际上，从某种意义上说，这总是……我们最近有点绕了个圈回来，但我们专注于许多不同的应用。当时的想法就是，让我们把在云端运行代码变得超级简单。可以是 GPU 代码、AI、机器学习，也可以是数据相关的东西。我们非常关注，你知道，如何扩展它？如何让开发者体验变得很棒？很早就意识到，要做到这一点，我们必须深入技术栈，构建在文件系统之上、构建在容器运行时之上，做了很多，你知道，解决了很多艰难的工程挑战。然后你提到了 chat、CPT，那其实并不算是杀手级应用。但如果你还记得，那年秋天，大约同一时间还出现了另一个东西，那就是 stable diffusion，而 stable diffusion 结果成了我们当时正在构建的东西的杀手级应用。那时我们只有四五个人。所以我们开始看到很多人来找我们，想要运行 stable diffusion 模型。然后我们真正……那让我们意识到，也许生成式 AI 推理实际上对我们来说是一个非常不错的用例。你是怎么发现的？我就是个强迫症式的推特用户。我现在更是如此，比如我现在太忙于建公司而分心，但我一直都很爱发推。我写了很多博客。所以我想我有一点粉丝基础。所以，你知道，在早期，很多推动增长的因素就是我自己强迫症式地发推谈论模型、谈论我在构建的东西，让人们兴奋起来，写长篇博客文章谈论云计算的走向等等。我称之为有点像创始人主导的营销。我总是觉得你多少得发挥你擅长的东西

7:30 而我擅长的就是，我猜就是发些垃圾帖、永远在线。所以，你知道，工作方面，仍然是这样，你知道，我觉得我们的很多增长就是我们在社交媒体上比较擅长，就是，你知道，经常在线。不错。多久之后你有了第一个客户？花了一段时间。另外，你早期融资了吗？我们有一段时间没有融资。实际上我自己出资了一段时间，甚至一直到最初的几名员工，从某种意义上说这还挺好的，因为可以对钱非常节俭，想要看到投资回报率。我很长一段时间没有拿薪水。实际上，到了某个时候这成了问题，因为我需要钱，但我们有一段时间没有融资。直到 2022 年初，我们才融了种子轮。所以那时我已经在这上面做了大约一年了。Akshah 大约五年前加入担任 CTO 和联合创始人，大约就是现在这个时候，八月，也许是 2021 年。但然后一直到大约 2022 年底，我们才看到任何有意义的收入。我想这就是我为你感到骄傲的原因。是的。然后 stable diffusion 在我们的案例中是主要的东西，所以我们从，你知道，2022 年 11 月左右的每月 100 美元，到下个月我们做到了每月 2000 美元。然后到 2023 年 1 月，我们突然做到了 3 万。你知道，所以终于我感觉这种巨大的、就像焦虑感从胸口卸下了。我想，也许这真的是我们找到了什么。感觉我们找到了什么。然后 2023 年我们增长得非常快。

9:00 我想从那时起我们基本上一直在快速增长。那个月入 3 万美元的时刻，是因为几个大客户，还是客户非常多样化？就一个。我不知道我有没有谈过那是谁，甚至他们自己是否知道，但我觉得他们有所怀疑。那是纽约一家叫 Mage 的公司。他们在做类似 stable diffusion 的业务。所以你知道，在某个时候，我想是 2023 年第一季度，他们大概占了我们收入的 80%、90%。但之后，你知道，收入开始稍微多样化了。2023 年底，我们迎来了 soon 这个大客户。他们也曾在一段时间内占我们收入的很大一部分。但后来，你知道，随着时间推移，我们现在的收入多样化其实相当不错了。是的。说到 stable diffusion 那些东西，我记得用它来跑 comfy UI。我猜很多人都在用 comfy UI，比如在上面跑 comfy UI 工作流。有些人是这样，但我觉得用模型的人往往更偏向底层的东西。比如他们做复杂的东西。他们用模型的方式，一直都很可定制。所以他们部署了自己的自定义工作流，带适配器和自定义推理，还有大量优化，而我们为了让开发者体验变得很好所做的所有东西，结果也非常适合运行生产工作负载。因为自动扩缩容其实是一个很相似的问题。比如你想让容器非常快速地扩缩容。所以我们恰好处于非常有利的位置，在早期承接了大量这类工作负载。自动扩缩容的工具当时不存在吗？比如为什么你得自己构建这些？

10:30 总的来说，我的观点是，你知道，每一个新的技术时代都会遇到工具的限制。你回顾一下，无论是 Web、云、移动端、PC，还是 AI 等等。所以我认为，情况往往是工具已经存在，但它们通常并不是特别好用。你常常需要重新思考很多这类工具，才能收获好处，真正实现涡轮增压般的加速，并开始为这个新世界构建东西。你在云上就看到了这一点，我们必须发明 Kubernetes 和 Docker 之类的东西。类似地，在生成式 AI 方面，你可以在上一代工具之上构建这些东西。你可以在 Kubernetes 和 Docker 等之上构建这些东西，但这些技术是为一种非常不同的工作负载而构建的。它们是为非常稳定的后端工作负载而构建的。它们的请求数量是平稳的，运行时的利用率相当低，因为 CPU 通常很便宜。所以人们不太担心成本。但现在我们进入了这个新世界，你有非常昂贵的 GPU，你想要管理容量和利用率。你有非常大的模型，需要通过网络加载到 CPU 再加载到 GPU。而事实证明，很多这类工具，它们根本不是为此而构建的。所以我们觉得，必须重建很多这样的东西，构建我们自己的，我提到了文件系统，但过去几年我们也花了很多时间，围绕 CPU 构建了一些非常酷的东西

12:00 快照、GPU 快照与审计，我们构建在自动扩缩容和调度器之上，还有各种容量管理工具。我们觉得你必须这么做，因为不能为这一代新软件在旧东西之上构建，不能为了做新东西而在旧东西上套一层包装。你需要稍微从零开始。所以我想问，自动扩缩容，你们在平台上有没有看到什么奇怪的行为？在你们扩容的时候有没有什么东西坏掉？我的意思是，容量时不时就是个挑战。比如你怎么弄到成百上千、上万块 GPU？所以这始终是个挑战。我们很早就不得不做多区域。所以我觉得事后看那是个关键决定，就是做这件事并且咬牙把它建起来。我们一直把容量管理看作一个大池子。就像全世界一个大池子。我们不把它看作每个区域各自运行的独立集群。所以这是我们早期做出的非常有影响的决定之一，就是那样去思考。所以我不知道有没有什么具体问题。我的意思是，我们显然有过 bug，有过宕机，我讨厌这样，我多么希望模型能百分之百在线，我觉得最近我们基本上达到了四到五个九，但在模型的历史上，当然我们有过一些撞上扩容问题的时候。是的。GPU 市场，其实在那之前。所以你们底层用的是 AWS、GCP 这些云厂商。

13:30 你们也会购买基础设施吗？不，我们聊聊云容量吧。因为我觉得这方面变化很大。比如头几年，我们几乎所有的容量都来自超大规模云厂商的竞价实例和按需容量，主要是竞价实例，运行在各个不同的区域，你知道，最初用的是 AWS，后来也用 GCP，再往后还有别的。我们很早就开始从 Oracle 租一些容量，因为我们想要 8 卡 A100 和 8 卡 H100，而且拿到了非常好的价格。随着时间推移，我们又增加了几家云，比如做预留实例，但直到去年年底，我们的大部分容量仍然是竞价实例或一年期预留。竞价实例就是你当场租用。没错。而且它还是可抢占的，这我们得应对。但这在 Agria 上还算行得通，因为你可以把来自基本上数百个区域的容量聚合起来。所以这个模式一直运作得很好，直到我们又开始扩张。我的意思是，首先，随着我们需要更多容量，Modal 开始增长得非常快，但根本上，云市场也开始发生很大变化。所以过去一年，基本上现在为了获得我们需要的容量，我们不得不与更多得多的云厂商合作。目前我们实际上与大约 25 家云厂商合作。而且在很多情况下，为了拿到我们想要的容量，我们不得不签三年以上的合同，并且要预付一大笔钱。而且你是在锁定三年期的价格，对吧？

15:00 所以，你知道，我们不得不花掉整整六到九个月的时间，重新思考我们如何获取容量。我们得建立一个完整的采购团队，基本上有四个人全职工作，他们做的就是采购容量、联系 Neocloud、谈交易、审核容量。我们还不得不建立一个完整的工程团队，大约 10 个人，专门管理来自世界各地的裸金属容量，处理网络问题、坏盘、GPU 掉总线等等各种事情。我们不拥有容量。到目前为止我们租用所有容量，通常都是三年期合同。这是一个重大决定。比如，我们应该买，还是应该继续租？我其实对此并不特别教条。我认为租用不错，因为它有一定的资本效率。如果你买，你知道，首先，没有那种运营复杂性。比如你得处理供应商和供货商，还有电力、线缆等各种东西。但还有财务复杂性。比如，你怎么融资？你应该股权融资吗，你可以融十亿美元去买硬件。那是一种非常稀释股权的方式。你也可以债务融资，但如果你这么做，你通常希望有锁定的长期收入来支付利息。这也不简单。但你知道，我们看经济账，我们看运营

16:30 复杂性，我们来看看风险。呃，这是一个复杂的权衡，而且，而且我们对，对，对收购持开放态度。我们会，我们会有点，还有其他的构建方式。就像我们，这是为了获得我们为2027年所需的能力。这需要大量的工作，在，在，你知道的，走很多不同的途径来，来锁定它。那么你是否需要雇佣很多像财务人员来做所有这些事情，还是不需要？真的吗？你知道，我不得不在这上面花很多时间，我们不得不，不，不，在这上面花很多时间。而且我确实认为随着时间的推移，我的意思是，有，有很多，你知道的，这些是非常昂贵的资产，呃，硬件。而且，你知道，我认为，我的意思是，这有点令人震惊。你开始看到像AI投资开始变成美国GDP的几个百分点，而且，你知道，开始看到像美国的利率，在，在最近几周飙升。而且我的意思是，我喜欢，在这个阶段，就像，你知道的，然后你去看看Neoclouds，就像这些是非常非常复杂的金融工程公司。所以是的，我们，我们看那个，而且，你知道，我们看所有的选项，而且，而且我们可能自己也不得不做类似的事情。我不想要像，我宁愿专注于成为，你知道的，像一家非常精简的软件公司，像做我们擅长的事情。我不认为软件公司拥有硬资产总体上是好事。你知道，就像，你应该拥有你的办公室吗？就像我不认为这对大多数公司来说一定是正确的解决方案。但当然，就像为了获得你想要的能力，就像，你知道的，一切都在桌面上。是的。而且我猜供应会在一段时间内受到限制。

18:00 所以你多少得这么做。比如，你，你同意这一点吗？还是你觉得这只是暂时的现象？我不一定同意。我的意思是，Nvidia 每个月交付几十万块 Blackwall，对吧？而且，我不想说市场是 100% 有效的，但总的来说，如果你愿意出高价，你就能拿到 GPU 产能，对吧？比如如果你愿意比别人出价更高，当然 Anthropic 和 OpenAI 也在尽可能多地采购。但根本上，市场上不断有供给进来。所以我不一定认为它是被完全垄断的资源。价格在上涨。我确实觉得这值得反思一下。硬件的常态就像脱离重力一样。最终，硬件价格总是会下降。所以我们现在处于一个有点奇怪的时期，硬件价格在上涨。而且，很难说这会持续多久，可能是再过六个月，也可能是再过 36 个月。真的很难知道会持续多久。但我根本上不确定说“你需要购买产能”是否成立，对我来说，这有点像疫情期间的卫生纸大抢购之类的。对吧。至少这像是挤兑银行之类的情况。但话虽如此，产能在一段时间内会紧张。而且价格在上涨。幸运的是，我认为这对我们甚至不一定是不利的。因为如果价格上涨，因为稀缺也意味着我们可以

19:30 向我们的客户收取更高的费用。而且，你知道，我们有很多预订以固定价格锁定，这在某种程度上意味着，我们实际上已经锁定了长期的 GPU 价格。所以我不太担心价格上涨。我更担心的是价格暴跌。我不认为这会在短期内发生。但我认为这对市场上的许多参与者来说会是相当糟糕的事情。为什么会糟糕？我认为，对任何锁定了大量供应的人来说都会很糟糕。是的。因为他们付了太多钱，他们付得太多了，你知道，所以存在买在顶部的风险，对吧？而且，就像我说的，我不太担心这个。我不认为这会像，你知道，短期内发生。但就像，你知道，就像我说的，从长期来看，硬件价格往往会下降。所以，这值得，你知道，反思一下，对吧？对。就像现在，我认为，我认为你的产能受到限制，这导致，你知道，用经济学术语来说，存在稀缺性租金，对吧？谁获取了稀缺性租金？我认为主要流向两个地方，主要是 Nvidia，但当然，也在某种程度上流向 Neoclouds，比如 Neoclouds，一般来说，你知道，任何拥有产能的人都有更好的经济效益。这在今天产能受限的情况下是无可否认的。长期来看，你知道，从非常长远来看，我不认为这必然完全正确，对吧？就像拥有 GPU 总能获得一点溢价，但这值得冒险吗？你知道，值得冒在资产负债表上持有这些非常昂贵的

21:00 资产的风险吗？我不太确定，但你知道，我们拭目以待，这需要几年时间才能度过。而且，从长远来看，你知道，我们可能会拥有一些，可能会租用一些，也许是组合，也许做一些其他疯狂的事情，联合导师，谁知道呢？也许一些复杂的 SPD，随便吧。嗯，一切皆有可能。你如何与 OpenAI 和 Anthropic 竞争并购买 GPU 产能？我假设他们想买下所有可用的东西。他们确实如此。而且，我认为你看到的是，嗯，你知道，很多，价格在上涨。这是对此的反映吗？话虽如此，你知道，NVIDIA 正在出货大量产品。我认为在某种程度上，NVIDIA 也有点偏好多元化。就像他们不想要，你知道，少量大买家，就像你看到的，你知道，超大规模厂商获得有限的分配。呃，我认为，你知道，NVIDIA 也更希望看到，你知道，许多 neoclouds 和许多替代云，获得更多分配。而且我认为，我们在市场上看到的一点也是，这更多是猜测，但我认为我们看到，你知道，规模较小、合作起来有点风险的云对我们来说是更多机会。就像，你知道，也许 NVIDIA，抱歉，Anthropic 和 OpenAI，我的感觉是，他们更愿意从，从，从尝试通过，呃，部分原因也是因为他们必须进行大量训练运行。所以他们需要非常大的块，但那里有大量产能，你知道，你可以在亚美尼亚获得 32 个节点。

22:30 在印尼你可以拿到 128 个节点。就像，你知道的，我们就喜欢这种玩意儿。就像，你知道的，所以我们可以去那边，在全球各地拿到很多，你知道的，更小规模的容量。价格相当不错。是不是基本上就像，如果有人给你提供一定数量的 GPU，那就是你唯一关心的？是不是没有别的，比如，哦，你关心它有多可靠，你知道的，是不是只是某个破仓库里有人放着这些东西？我的意思是，有时候确实是个有点破的仓库。感觉就像是，你知道的，但是，但是，我们基本上确实有一份规格表。比如当我们和供应商谈的时候，我们做的第一件事就是发给他们一份规格表。我们说，我们需要这个数量的 IOPS。我们需要，你知道的，比如 CPU 至少要，至少，你知道的，要有，要有这么多，你知道的，不管是什么 pass mark 分数之类的。我们需要看到，你知道的，最低的，你知道的，北向、南向带宽，出站和入站。所以我们确实有一份需求清单。我们经常尝试做一个验收测试期，就是拿一个节点，然后运行并随时间测试，你知道的，在某种程度上，这也需要一点盲目的信任。你知道的，有时候我们会签非常大的承诺，你知道的，几千万或几亿美元，然后我们开始和一个我们还不完全了解的供应商合作。而且，你知道的，随着时间推移，比如，你知道的，我们会看到的，你知道的，而且，而且，而且我觉得，过一段时间，比如，有一点，有一点那种，你知道的，我们自己的那种，比如，你知道的，就像 SemiAnalysis 有 ClusterMAX 一样，我有点觉得我们内部几乎有自己的，比如 ModalMAX 评分。比如，我知道我比其他人更喜欢哪些供应商，你知道的，比如，而且，而且

24:00 我们倾向于与我们见过的供应商做更多交易，你知道，他们非常可靠，不只是可靠，还有当我们遇到支持问题时，他们是否非常响应，我们要求的修复问题的周转时间如何。如果出故障了，你知道，他们多快能修好？诸如此类的事情。你同意对智能的需求基本上是无限的吗？如果是这样，为什么不尽可能多地购买容量？然后，如果你买多了，你可以直接卖给别人。我觉得当存在稀缺资源时，很容易说，我们就尽可能多买。但我认为，你知道，首先，如果我们今天买了，我们无法全部利用。我们可以利用很多，对吧？但你可以把它卖给其他想用的人，对吧？我认为挑战在于，如果市场上发生了某些事情，会有很多相关性同时发生。如果价格下跌、利用率下降，转售容量的市场同时消失。我坐在这里，非常激进，每天购买容量并筹集资金来购买，你知道，所以总体而言，我非常看涨，但我想指出，你知道，这里有尾部风险。当这些尾部风险发生时，可能会变得非常糟糕。部分原因是，也许我老了，我过去见过这些，我实际上在1999年有第一份工作。所以我看到了互联网泡沫危机，也看到了金融危机之类的事情。而且，你知道，我想建立一家能存在20年的公司

25:30 到 30，再多 50 年，对吧？所以，你知道，我认为我们现在当然可以出去疯狂地购买大量产能，试图垄断市场。但是，你知道亨特兄弟吗？这是交易界的一个传奇故事。他们试图垄断白银市场。结果惨败。你知道，所以垄断市场，我认为总的来说，这可能不是一个好主意。但是，但是我们总体上非常、非常激进，但我们正在加大投入，你知道，按照一个激进的计划，基本上匹配我们的需求增长，匹配我们想要在明年年底达到的算力储备或运行规模。你们只买 NVIDIA 的东西，还是也从其他类型的公司购买产能，比如训练它们，我们正在考虑。我认为替代加速器开始变得非常有趣。你知道，讽刺的是，过去五年，甚至可能十年，我每年都在说，你知道，明年将是非 NVIDIA 加速器之年。然后我总是错的。但是，但是我真的认为 2027 年可能是非 NVIDIA 加速器之年。终于，所以我们会看看今年我是不是又错了。但是所以我，你知道，我，这些在某些方面开始获得非常好的分数。我认为，你知道，TPU 显然在 Google 内部被非常广泛地使用。似乎运行得非常好。我对训练了解不多。我不觉得我看到那么多，但是，你知道，团队非常优秀。我知道他们正在做很多软件方面的工作。

27:00 你知道，硬件在某种程度上确实很难。软件相对容易一些。这让我觉得，你知道，如果我们只是为这些加速器修好软件层，我觉得它们会有很大的需求。而且我认为从根本上说，你知道，既然 NVIDIA 是这么大的瓶颈，我们可能会看到更多替代加速器。话虽如此，你知道，管道更下游还有瓶颈。我是说，你知道，还有像 TSMC 和 ASML，也许那才是这一切更深层的瓶颈。但是，你知道，长期来看，我认为垄断并不总是持久。所以我认为市场有更多替代方案会是健康的。尽管我很喜欢 NVIDIA，这一点要说清楚，我是说，他们显然是一家了不起的公司，产品也很棒。我确实也预期市场会出现更多元化。还有，你知道，像 Cerebras 和 Grock 这些东西。是的。你对其中一些供应商有什么看法？我觉得它们非常令人兴奋。而且，你知道，你能看到它们在某些工作负载上表现非常好。而且，你知道，有一些有意思的人在做这种混合的、你知道，分离式推理，他们用 NVIDIA 做预填充，用 Cerebras 做解码之类的。哦，我其实还没听说过这个。是的。还有 Grock 也是，你知道，因为很多东西，你知道，有些东西是计算受限，有些东西是内存受限。所以你可以 sort of 混搭。我觉得这相当令人兴奋。在实践中部署这些东西时会变得非常复杂。但我确实认为有很多、很多，你知道，机会，以及这些 sort of，你知道，更复杂的硬件配置。你从这些 Neoclouds 那里有没有学到什么，比如，我不知道，像 Nebius 之类的，以及它们做事的方式与

28:30 就像你们在做的事情？我觉得关于 Neoclouds 非常有意思的一点是，它们怎么能扩张得这么快？实际上我觉得最疯狂的例子就是 N scale，它从默默无闻，到拥有，你知道，随便吧，数千亿美元的项目储备，尽管几乎没有多少产能。Neoclouds 怎么能增长得这么快？我的意思是，基本上答案就是，你知道，这可能很明显，但它们做的事情就是，去找像 Anthropic 或者 OpenAI 之类的大买家，然后基本上就说，嘿，你想不想买五年的芯片？签这份合同，我们给你一个好价格。然后他们拿着这份合同去华尔街说，看，我们有了这个大买家，他们承诺了价值 100 亿美元的芯片。我们现在只需要把它变现。然后他们就去筹钱。接着他们去找 Super Micro 和 Dell 之类的，拿到硬件，然后安装。然后现在他们就把货交付给客户了，对吧。所以你有了这种非常巧妙的金融结构，基本上可以预售你甚至还没买的东西。这意味着你可以极快地扩张。所以这就是我们在这些 Neoclouds 身上看到的，它们能非常好、非常快地扩张。我觉得 Modal 因为一直更偏向 PLG 模式，而不是那种能锁定长期合同的能力，所以从财务角度看，要那么快扩张会更有挑战一些，尽管我认为，从长远来看，我们会达到那个

30:00 同一个点。但我觉得，观察这些 Neocloud 尤其是像 CoreWeave 和那些大公司，是件很有意思的事。他们在如何融资、如何把 GPU 当作贷款抵押品等方面非常非常精明。那个相关性是什么，我忘了是谁，但好像有家金融公司说，哦，我们要做，哦，我想是 Nvidia。实际上，他们说，我们要把 GPU 变成一种资产类别。这是不是只是因为他们需要融资，想弄清楚从哪里弄钱？这有什么相关性？我是说，有一堆初创公司。我不知道你有没有想过这个。有像 Silicon Data 和 computer exchange，还有像 Orn 等等，还有另外几家，基本上是在尝试创建价格指数和指数，交易衍生品之类的。我觉得这很有意思。就像，我觉得对于算力来说，确实，算力作为一种资产类别存在许多挑战。它绝对不像其他资产类别那样同质化。你有不同的网络、不同的位置，还有不同的磁盘、CPU 等各种东西以及可靠性。我的意思是，话虽如此，你也可以对石油说同样的话，而我们某种程度上把石油变成了相对商品化、可交易的资产。这还有待观察。我确实认为，在我们目前 AI 数据中心和 GPU 部署的规模下，我们确实需要在风险对冲、如何分担风险等方面变得更聪明一些，而且，你知道，

31:30 更像是保险之类的东西。所以，我非常高兴人们对此有很多思考。话虽如此，我不完全确定我们是否会看到，你知道，GPU 变成某种，你知道，不管是什么，类似 Henry Hub，你知道，布伦特原油那种，你知道，超级商品化，像可交易资产一样。是的。我想把话题转到 modal sandboxes。是的。我想在那之前，modal 有哪些不同的产品？我想主要产品显然是 GPU。然后你有 modal sandboxes。还有别的吗？我认为我们有的两个大产品基本上是：第一是 modal functions，第二是 modal sandboxes。所以，对于 modal functions，我们在 2021 年推出的。当时的想法是像无服务器函数，非常容易部署，非常容易扩展，我们管理所有容量。结果证明这对推理非常非常好，对管理 GPU 和运行之类的东西也非常好。但我们实际上从来没有过，你知道，专门的推理产品或 GPU 产品本身。我们从未把 modal 视为 neocloud，我认为有被吸入那个方向并成为 neocloud 的风险，就像只是卖，你知道，我认为是矩形的东西，就像这是一个三年。你得到这么多 GPU。Models 一直非常擅长，就像你部署一些东西，你可以扩展它。你有这种非常周期性或突发性的东西。我们管理所有容量。美妙的是，因为我们有成千上万的客户，

33:00 我们可以把这种容量聚合起来，获得非常平滑的工作负载，你知道，非常平滑的聚合需求，我们可以非常高效地管理它。所以 modal functions 是核心。我们在 2021 年推出了它。就像我提到的，我们在 2023 年开始看到产品市场契合。而现在，你知道，它大约占我们收入的三分之二，并赋能了很多，你知道，在实践中，它主要是推理，但你可以把它用于各种其他用途。Modal sandboxes，我们在 2023 年推出了它。实际上，你知道，它花了很长时间才看到产品市场契合，类似于 functions。我们为什么推出它？我们意识到我们交谈过的很多人都在使用 LLM 输出代码。这就像我和 Akshar 某天午餐时聊天。我们说，也许他们需要一个地方来运行那段代码。然后我们意识到，实际上，我们其实已经拥有了很多基础设施，可以直接接收代码并运行它。因为我们已经构建了用于管理容器和隔离的原语。我们已经有，你知道，G visors 和隔离原语之类的东西。所以我们对这个想法有点兴奋。Akshar 写了一个原型，基本上在一天或两天左右就推出了。零。就像我们看到的增长非常小。我不想把它说得微不足道。因为我们确实看到了一些增长。就像有某种，你知道，而且那是 2023 年。所以是的，那是 2023 年。而且就像，你知道，有些人采用了它。我们有一些早期用户，他们在某些事情上很喜欢它。但总体来说，就像在进入 2023 年夏天时，它就像是一个相当微不足道的收入来源。所以两年后，我们甚至讨论过，我们到底还需要这个产品吗？

34:30 我们应该关掉它吗？然后它就开始爆发式增长了。所以是2023年。我认为2023年真正发生的是，LLM在生成代码方面变得非常擅长。2023年。2023年。我的意思是，大概就是那个时候。感觉就像是从能够重构函数、编写代码片段，变成了能够生成完整的应用。对。然后我们开始看到大量增长来自vibe coding，你知道，像Loveable这样的公司和其他一些公司。然后，你知道，像Ramps这样的公司开始将模型用于后台代理。接着我们开始看到来自许多Neolabs的巨大增长，他们在模型沙箱上做强化学习之类的事情。所以沙箱现在大约占我们收入的三分之一。你知道，我们基本上把它当作我们的第二个产品。对我来说，这真的很令人兴奋，因为即使在模型早期，我一直知道模型是关于打造一整套产品工具包的。就像我们一直想成为几乎像云一样，你知道，你有存储和计算以及一堆不同的使用方式。我们一直想成为多产品供应商，但直到现在这在实践中才真正实现。我觉得现在这已经相当清楚了。我们有函数，我们有沙箱。我们还在做一堆其他东西。我们在做训练，在做批处理，我们还在做一堆其他东西。所以对我来说，这种演变成多产品公司的过程，我觉得非常棒，因为这一直是愿景的核心。即使在最早的演示文稿中，我也有很多关于这方面的幻灯片。但直到最近，这对外界来说才真正成为现实。

36:00 你们有没有在做的项目，你认为会成为下一个这样的产品？或者对市场有什么预测吗？诸如此类的问题？我是说，我非常看好训练。我认为训练有点像是一个尚未被充分服务的市场。显然，大家对超大规模训练运行有很多关注。但我们实际上看到了大量需求，来自那种较小和中型的训练细分市场。所以我这周晚些时候会稍微谈谈这个。我们正在组织一个会议。Batch 我认为是一个很有意思的市场。我觉得我们看到了很多需求，来自那些需要……你知道，我们前几天有一个客户，需要转录，你知道，5000 年的音频。他们有，你知道，数以百万计的音频文件。而我们有很多容量。所以我们让他们做了这件事，你知道，花了几天时间。我认为那里有一些非常有趣的用例。非常看好低延迟的东西。我认为会有很多，你知道，一旦我们让音频，比如语音到语音模型跑起来，比如语音模型。我有点预期那会是一个非常有趣的市场。然后当然，还有所有这些 agent 相关的东西，对吧？而在某种程度上，我对 agent 这个词有点复杂的感受，因为我觉得在某些方面，我认为所有软件都会变得 agent 化。所以对我来说几乎就像是，agent 化到底意味着什么？我觉得所有软件在某种程度上，都会随着时间变成 AI 驱动的。但显然，有一组构建模块，你知道，用来做更多 LLM 驱动的软件，这似乎，你知道，在所有不同的用例中都是共通的。

37:30 我们是这样想的，你知道，我对模型的看法是，我们做的是构建积木的生意，你知道，我们想给你提供令人惊叹的、可泛化的、经过实战检验的、你知道，通用的、可组合的积木。而且我觉得还有一些非常有趣的东西，比如，你知道，在智能体用例方面。这很有意思。你提到训练，因为我觉得在另一档播客里，大概一年前，或者两年前，你说过你并没有特别专注于训练，市场发生了什么变化？我的意思是，我们多年来断断续续地在构建一个训练产品。所以我不会说我们，好像我们从未专注过它。但这就像，就像那些事情之一，我们就像，我的意思是，现在我们大概有150名工程师，但是，你知道，一两年前，我们的工程师人数要少得多。所以，但正是因为，你知道，规模更小，我们不得不稍微更专注一些。于是推理就成了我们唯一的焦点，并且在很多年里都是我们唯一的焦点。但我们现在更大了。就像我们，我们，我们确实有机会在其他市场也下注。我也非常感兴趣，你知道，就是继续这个清单。后训练、强化学习。我觉得这周围有一些非常令人兴奋的东西。我觉得还有很多其他东西，比如围绕存储、构建数据仓库和特征存储、实时流式管道以及各种，我不知道。我觉得对modal的长期看法基本上就是，你想构建一些酷炫的AI东西，你知道，我们都能帮你搞定，你知道，我们有所有的积木，比如comp任务，而且我们会帮你，你知道，把所有这些积木组合起来。

39:00 这就是软件的美妙之处：只要有合适的构建模块，你几乎可以构建任何东西。所以，这正是我们想要打造的公司和产品类型。当你说训练时，你更关注的是那些进行大规模训练的大型企业吗？不，不，他们会去做那些超大规模或Neo云的大单，拿到10万块GPU之类的。但在我看来，像研究人员迭代或训练较小模型的人群，这个市场有点被服务不足。还有德州海滩和语音、文本模型，计算生物技术实际上是我们看到大量需求的市场。我们在那个领域有很多客户，训练相对适中的模型。你可以在64块GPU上训练，或者推荐系统，很多传统机器学习。有很多东西你不需要10万块GPU。那个市场，我认为我们暂时不会专注于此。但我认为在稍小的市场中，有很多东西相对服务不足。搭建训练集群非常困难。是的。你有没有注意到，比如从GLM 5.2或可能Can we 2.5开始，很多人都在使用并微调它们？如果说有什么不同，我实际上看到了相反的方向。我觉得几年前微调的价值主张是：开源模型没那么好，但如果你拿一个小模型在你的特定

40:30领域微调，你实际上可以在质量和成本上达到前沿水平。今天，我会说开源模型已经非常好，你基本上可以直接替换运行，而不是用前沿模型，你可以运行Kimi K3或DeepSeq 4.1或GLM 5.3。它们几乎和前沿模型一样好，成本却低得多。所以我们仍然看到大量后训练需求。我确实认为这是一个非常有趣的空间，我们正在该领域开发一些产品，包括SFT以及强化学习等。我还想说，微调不再是你必须做的事情。它非常可选。开源模型开箱即用也能工作得很好。回到沙盒。我忘了问你。你们提供的沙盒需要更新什么，而现有沙盒不允许？没有太多。我认为我们一直非常擅长的事情，我是说，我们是infrageeks，对吧？我们非常擅长扩展和可靠性以及做很多事情。我认为这在很大程度上解释了为什么我们拥有，在我看来，我没有数字，但我认为我们在这个领域拥有迄今为止最大的市场份额。我们只是非常擅长运行数十万甚至数百万个沙盒。是的。我得说，为了处理那种规模，我们不得不重写我们的系统很多次。

42:00 但我们一直都很擅长这个。还有其他一些事情，随着时间推移。我是说，总的来说沙箱领域，你知道，它并不是最有差异化的领域，但你知道，我们正在做一些非常酷的东西。比如随着时间推移，我们不得不添加对 VM 的支持，比如虚拟机。所以我们之前是在 GVisor 上构建我们的沙箱产品，但现在我们也提供完整的虚拟化机器。事实证明，对于某些用例，你确实需要完整的 Linux 兼容性。我们还必须构建其他东西，你知道，更好的原语，比如更交互式的，你知道，在沙箱之间来回传输数据，处理文件系统。我们不得不在快照上花了很多时间。我认为从长远来看，很多会形成差异化的东西将会是，你知道，非常酷的东西，比如，你知道，分叉沙箱、快照、恢复快照，比如实时迁移。总的来说，我觉得沙箱，沙箱的新颖之处始终是，你知道，这种短暂性，比如你可以在几秒钟内启动沙箱。这是我喜欢的东西，这是我想进一步挖掘的一条线索。我认为你可以做很多非常酷的事情，你知道，把 agent 和 harness 解耦，把它与沙箱解耦，让 agent 控制许多沙箱、分叉沙箱，你知道，对东西做快照，比如，你知道，基本上运行非常短命的，也许每个命令一个，像这样的事情。所以我认为那里有很多很酷的东西。我们已经花了很多时间做这个，你知道，围绕快照。但我认为那里也有很多很酷的东西可以构建。你几次提到了快照。

43:30 Like what do you see as like the use case for snapshotting? Like what would that give us that we don’t get right now? I think the big thing for me is like today a lot of sandbox workloads are kind of like lift and shift. Like people think a lot about sandboxes almost like, you know, I have my dev box, now I’m going to shove it in the cloud, I’m going to have an agent control it, maybe I’ll run the harness inside the sandbox. I don’t know if like long term that’s like the right way to think about, you know, on like compute for agents. I think a much better model would be you’re going to have this like very long running agents, which almost like to me are like kind of durable execution workflows. Like they’re going to run, you know, 24 seven, you know, every once in a while they’re going to run commands and, and you know, having the ability to like spin up a sandbox in, you know, 100 millisecond, run a few commands and then like shut it down and like snapshot it. I think the faster you can get that like, you know, wake up, you know, go to sleep cycle, the more you can sort of create sandboxes that only run for like a very brief period and that will make compute much more efficient. And it will also enable you to do some really cool things, you know, like forking and moving things around and, and, and, you know, having like imagine, you know, like Claude code instead of, you know, having sort of a very linear flow, like sequential flow, like kind of branching and, you know, running hundreds of agents at the same time. I think a lot of like CI is like another like illustrative example, like for most of the, the history of CI, you know, talk about like GitHub actions and things like that.

45:00 Like you had a very sequential flow, like, you know, you slowly install a bunch of stuff, then you run the test suite and then, you know, generates some reports or whatever. And it’s like, you can break it up into shards, but it’s kind of annoying. Like what I would love to do is like take your entire test suite and just run every single test in its own sandbox, right? Like, and just paralyze the shit out of that around like a thousand sandboxes. Why can’t that be done? I mean, the problem is like there’s too much, there’s too much overhead around like, you know, starting and stopping sandboxes. So I think the more like you create like, you know, frictionless, like starting and stopping, the more you can enter a world where like you can do this like extremely bursty, like large scale, like run as much as you think you want in parallel. So, you know, I’m an infrageek, like I said, like I get very excited about these things. You know, once we figure out all these primitives and the right building blocks, I think it’s going to unlock a lot of really cool stuff that, you know, we previously could not do with software. Why hasn’t that already been built? I would assume even previously people would want to parallelize as much of their CI as possible. You can’t, but the overhead, it’s like Amdahl’s law or whatever. Right? Like the overhead of parallelization right now is very high because like right now, like you can get up actions, for instance, like we parallelize in the modal test suite, right? But like what that basically means is like now if you take your test suite and you split it up in two, like now each of those shards both need to do all the installs and the test setup. And that takes a long time. Like it can take minutes, but let’s say you can do the test set up once

46:30 and then you snapshot the container and now you clone it and now you can run the test suites independently. I think a lot of the benefits cannot be, you can’t materialize those benefits until you get the friction and the overhead down, the transaction cost. I think you guys have a partnership with Anthropic and they’re using sandboxes for something, like modal sandboxes. Yeah, I mean the managed agents with Anthropic, this is sort of bring your own sandbox integration. So you can use modal for those, yeah. How did that come out? Like come to be? I think Anthropic is like trying to create like a thriving ecosystem and community and so I think they make money from inference, right? Okay. I think it’s very much in their interest to promote, for everything else, as much of an ecosystem as possible and they want to have the largest possible surface area as long as you use Anthropic for inference. So I think it makes like very rational sense for them to partner with us as a sandbox provider. Also, I feel like we have to bring up the hugging face incident. Are you okay talking about that? Yeah, yeah, I mean it’s all there. Yeah, yeah. So yeah, could you just briefly describe like what happened and I think they were using the modal sandbox but it wasn’t your, you know, it wasn’t like a fault of the sandbox. Yeah, exactly. I forget all the details of the exploit but I mean, you know, I think what you’re referring to is the fact that like modal was involved. Yeah. And so, you know, just to comment on that, like basically the agents running inside,

48:00 you know, that exploited hugging face, they discovered a vulnerability in an app that was deployed on modal and they used that to sort of, it’s almost like a springboard for getting access, getting full internet access is my understanding. I don’t remember the full details. To be clear, this is an app running a modal that was hacked. Like it was not modal that was hacked. So it’s like AWS, a lot of people run stuff on AWS, occasionally those apps get hacked. Does that mean AWS got hacked? Like no, of course not, right? And so there’s a lot of people running apps on modal and, you know, one of those apps got hacked. And I think, but like there are some lessons we can learn and I think one of the things we’ve been reflecting on is like, could we have more sensible, you know, security restrictions by default, I think to use the AWS analogy and just as another analogy, you know, S3 used to be this, like, you know, it used to be very easy to accidentally make an S3 bucket public and AWS realized like, you know, it doesn’t necessarily cause it, you know, poses security problem for AWS, per se. But a lot of their customers accidentally would, you know, like leak data. And so S3, it’s much harder now to make S3 buckets public. And so in a similar way, it’s like, we’re exploring like, you know, can we make security a little bit more restricted by default without necessarily, you know, destroying the magic that makes modal good. And I think there’s also, you know, it’s kind of a separate sidetrack is like, agents are getting really good at cybersecurity. So we’ve actually also, as a sort of, you know,

49:30 separate sort of learning from this incident, like we started using agents internally quite a lot to try to find vulnerabilities so far hasn’t found anything, but like, they’re very good at like, probing into like very deep parts of the system and exploiting, trying to exploit things and trying to escalate and get out of sandboxes. So, you know, it’s, it’s, it’s something we’re exploring very actively right now. How exactly do you get the agents to like run through the code base? You’re just like, hey, like find security vulnerabilities? Well, no, you just like, you just put them in a sandbox and tell them like, can you try to explain? Just run through the escape from the sandbox. It’s like, escape the room, but for agents. Cool. When did you start doing that? We started doing it a few months ago. And it’s pretty exciting. Like when Fable, I guess, came out. Yeah, roughly around the same time. Okay. I mean, my view on like cybersecurity is that, you know, I actually think they’re very good at finding vulnerabilities, but I think you can also use them defensively very well. So I, I, I think, you know, companies like modal and many other companies right now, like, I think, you know, if you go to talk to like pretty much every system, I’m sure they’re all thinking about the same thing. Like how do we use agents defensively to, to, to, you know, fix all our vulnerabilities? So over time, actually, in a way, like I’m not that nervous. I think if anything, like agents and cybersecurity, all the breakthroughs in cybersecurity will, will make the world safer, if anything. Do you think that like all of the cybersecurity, the big cybersecurity companies are just cooked? I don’t know if that’s true, because I mean, you’re asking something that’s like very far

51:00 outside of my like core expertise. But when I think about like cybersecurity, a lot of what you’re selling is effectively fear. And that’s almost like a branding and like marketing strategy, as much as, you know, obviously the core products. But I think, you know, brand and, and things like that are actually like arguably like more valuable in the air of AI. So I wouldn’t necessarily jump to the conclusion that they’re cooked. I think they have, you know, deep relationships with the enterprise customers and they have, you know, big sales teams are very good at telling, you know, spooky stories and, and things like that. And I think they’re going to continue to be very good at that. I just ask, because like, yeah, you’re doing it yourself, you’re not going to one of like the Palo Alto networks or whatever. But I guess like it’s only like giant enterprises that care about that. Also, they can blame somebody if something goes wrong. That is true. Yeah. Yeah. You kind of want to hire in many cases, just like an established vendor in order to blame them. I mean, to some extent, it does reduce a little bit the need for us to do, to bring in pen testers like human pen testers, which most startups like do on a regular cycle. So maybe previously, you know, we would do it, I actually don’t know what the cycle was, but, but I think we used to do it like once every six months, roughly. Maybe now we only need to do it once every year. I don’t know. But, but like, I think with all of these things, you know, humans can also use them to get more efficiently. So like, you know, like, I don’t know, I’m kind of a believer also that like humans with AI is even more powerful now. So like, maybe there’s an argument that like we can bring in pen testers

52:30 who are really good at using agents and, you know, find vulnerabilities even faster. I don’t know. I’m not that, you know, I’m an optimist. I started a company. So like, you know, I’m generally like optimistic about everything. But like, I certainly don’t think humans are cooked or companies are cooked. Like I think in the era of AI, a lot of companies are going to figure out, you know, how to use AI to be even more efficient. I want to transition a little bit to like developer experience, because like early on, I mean, even now, like you guys focus a lot on that. I mean, you’ve talked a lot about that. Does developer experience even matter anymore? I think it still matters. And I think you’re sort of, you know, like the context, I think you’re official, like the thing you’re talking about is like, now with agents, right? Like the, the developer experience, like the docs are already there. The agents are going to look at the docs, like all that stuff. To that point, though, like almost all the things that make developer experience good, also makes the agent experience good is kind of what we’ve seen in practice, which is like things like having good docs, having descriptive error messages, having fast feedback loops, right? Like, I do think there’s certain things have changed. The other day, I was like setting something up with Cloudflare, and it’s like, I find Cloudflare like notoriously annoying to work with sometimes. And, and I threw an agent at it, and it was able to like just configure a bunch of like, you know, things and pollute me for me. And, you know, and so I will say like agents does, they do make it a little bit easier to work with things that are like harder to work with in the past, you know, like the sort of human toll of dealing with the AML as arguably gone down a little bit, because you can just like,

54:00 you know, tell your agent to deal with it. But, but for the most part, I think, you know, the things that make developer experience good also make agent experience good. I think to some extent, also like infrastructure over time, anyway, becomes, it needs to be more programmatic. I think that’s always been the thesis about sandboxes, is that, you know, like things that you previously had to configure using YAML, that should just be an SDK. That should be more like imperative, like launch a sandbox, like do stuff with it, tear the sandbox down, right? Like, rather than sort of declarative using, using, you know, some sort of DSL, it also pushes like a little bit of a different interface that maybe previously didn’t exist. I think in either way, like SDKs are as relevant as they were, as they always were, if not more. Is there anything you changed in your product? Just because like, you know, most people are just running agents on it? We do. I mean, we also, we actually do run a lot of internal tests where we have agents, we tell agents to like, build a mobile app and run it. And I mean, a very concrete thing that is kind of funny is that we have actually seen that like, because of agent training data, it’s very hard to deprecate things now. We see that like, agents often like, love using, you know, old versions of modal syntax and it’s kind of harder to nudge them away from that. So we’ve actually had to be a little bit more conservative going forward, like, you know, over the last year or so with, with how we deprecate and remove features, it takes, we have to be very careful to update every single example that’s out

55:30 there online. Yeah. And, and, and, you know, skills, right? Yeah, there’s like stuff like that. But, but like, there’s also an insane amount of like modal code that’s now like online too that goes into the training data. I’ve seen like that to be like kind of a, you know, a little bit of like an interesting example where like, another thing by the way too is like, there’s obviously like to sort of a similar to like search engine optimization, there’s like agent engine, you know, whatever people call it, agent optimization. We often like, run internal benchmarks, we’re like, ask an agent, like, hey, like, can you deploy a stable diffusion for us? Use whatever tool you want. And we see if it recommends modal or not. And those things are very useful for us to sort of track over time. Or like, hey, spin up some sandboxes or like, hey, like rank all the sandbox vendors. So we spent a lot of time like doing those things to make sure. What do you do about that? Yeah, you say, hey, like, you know, build this stable diffusion thing, does it recommend modal or not? Like, yeah, what do you do about that? Yeah, I think that’s harder. I think, you know, it’s like search engine optimization, I think it’s primarily through content is like, you just have to put out a lot of content. You can also to some extent, like sort of see the perception, I think, a little bit more like you can ask an agent like, you know, how does, you know, rank all the sandbox vendors by reliability or rank all the sandbox vendors by price and sort of see, and it does feel like agents are a little bit more prone to like, misunderstanding pricing and, and, you know, things like that. So we also spend a lot of time like just making content more clear

57:00 and more like, you know, understandable for agents, because we know they’re all indexing, like everything that’s out there and that’s talking about modal. Have you thought about just like, making a bunch of GitHub repos and having the agents like write a bunch of modal code in there? Have you tried that? Slop cannon. I think the truth is like, that is probably something you can do. My guess is like LM training data today is probably quite insensitive to like, what’s like AI output in the first place and you can probably like create a lot of slop. Like we thought about it, like, I don’t love that idea. But like in theory, like much like, you know, SEO in the early days and still to some extent, like auto-generated content is like a very effective strategy. I think it’s possible. Like, you know, you could, you can do those things too. Yeah. I want to switch to FDEs. Apparently, you guys hire like forward-appoid engineers. Like, what do you do? We’re in a new era of like, building applications and, and, and, you know, things that we couldn’t build, you know, just a few years ago. And we see a lot of companies that, you know, they need a little bit of help, right? Like, you know, and so we started investing in FDEs a couple years ago, but it’s really been this year, it was kind of scaled it up a lot. I think we started this year having like four FDEs now. I think we have like 24 or something like that. So we’ve grown it a lot. And they do it, you know, it’s more, it’s not exactly the pound term model, but they do a lot of like heavy, you know, sort of customer engagement. Like they work with a lot of customers on performance tuning, especially with LMS. We see that there’s so much benefit from like, you know, sitting down

58:30 with customers and training draft models and tweaking, you know, parameters and DLM and SG Lang. They often, you know, fix bugs or not fix bugs, but like, you know, when customers run into issues, like they often work with customers trying to figure out like, how do we work around that? Or another thing we’ve used FDEs a lot for is like, you know, kind of proactively, you know, if we see certain customers, you know, over provisioning with model, like customers requesting too much CPU or too much memory, for instance, like that’s kind of expensive. And like, we try to catch those things and we try to sit down with those customers like, yo, like you’re spending a lot of money on model right now, we can actually help you save money. And then of course, like with sort of active deals, you know, we helped them migrate code from other platforms, things like that. I think, you know, we never have the intention of becoming like service heavy business. I think that the goal for FDEs is like, you know, A is to be a part of, you know, the sort of sales process and like closing deals. B is to serve customers and do some like, you know, customer support. But like, I actually think like the most important thing is like, C is like bridging gaps in the product that we later want to turn into like real features. So I think to a large extent, you know, anything an FDE does, you know, my question is always like long term, how do we turn this into product so we don’t need an FDE for this? Yeah. Do you guys hire PMs?

1:00:00 Product managers? Yeah. No. Would you say these FDEs are kind of like pseudo ish PMs? It’s a little bit of a different function. Like I think certainly on the more like, you know, customer facing side they are, but there’s like, you know, on the more like inward, you know, internal side, like this is sort of separate part of like product management, which is more like, what are the next building blocks Modal should invest in and like, you know, and then there’s like something in between like, you know, looking at like maybe 10, 20 different customers that are all asking for like slightly different things, but maybe what are the patterns? Like how do we turn this into product? We do have people doing product with us and it’s sort of evolved, you know, we’ve had people basically like become product managers after previously being either DevRel or engineers and showing strong aptitude on doing thinking very strategically about what we have to build. So I’m actually not like anti product management. I do think that a lot of companies in the past hired product management product managers too eagerly. And I think there’s a risk that you kind of, I can sort of see it in a lot of products like certain products like you can tell like, there’s just like eager product managers that just thought of like, kind of like, wigity things and just ship them. I think doing product management at an infra company really well is very, very hard because I think it takes like a very deep, you know, level of strategic thinking thinking about like, what are the foundational building blocks? Where do we want to be in one to two years? Because the truth is building infrastructure is very hard. And how do we make sure we don’t end up with like 35 features that are like,

1:01:30 kind of similar looking, but all do like a little bit slightly different things. That sort of level of thinking is something that, you know, is very rare to find. And for that reason, we haven’t aggressively tried to build up a product management team yet. Yeah, that makes sense. How would you say your teams have been evolving maybe like since Genialite? Because I would assume, yeah, I mean, you obviously worked at like Spotify, you had like a certain team structure, have your team structures like evolved a lot now that, you know, we have coding agents. In a funny way, like, yes, I do think team structures have changed a lot, but I’m actually not convinced it was like Genialite driving that. I think in particular, like one thing I’ve seen a lot is that there’s been sort of a focus, like a swing back towards being in IC that I actually think is very, very positive for the space as a whole. I think, you know, if you look back at the space like 10 years ago, 20 years ago, everyone wanted to be in management because that was the way you would get more money and more recognition and more prestige. I think that shift actually started even before Gen AI, but I do think it’s like heavily accelerated with AI. Like, you know, we make it very clear to everyone at Modal that like manager or RC, it’s like there’s no difference in prestige. Like everyone is just a member of technical staff. You know, it’s about ultimately your contributions and we see a lot of X managers, you know, becoming ICs and because they want to go back and build code. And so, and you know, and there’s been sort of shift away from like, you know, non-technical managers,

1:03:00 everyone, including me, I still write a lot of code at Modal and Akshath too and pretty much everyone in the tech team and a lot of people who are not in the tech team still write code. So that to me is like, you know, a recent phenomenon, but I actually don’t know if it’s like necessarily cost by Gen AI, but I think Gen AI has like greatly accelerated it because it made it even lower the cost of like writing code is even easier. So if you have like non-technical people like committing code, like how do you, how do you review that? Like how do you guys do code reviews? We do code reviews, but it’s, we think of it as optional. I mean, for most, my view is like code reviews in large part is like people, in many cases, trying to just escape the accountability of like, you know, merging things into production. And if you have that like, you know, if you have that culture, like then to me, that’s a sign that you need to invest way more in, you know, CD and CI and things like that is CI in particular, but also CD, containers integration, because it’s deployment. We do code reviews. Like, I mean, like when I write code, like occasionally, like I’m asking, you know, authors, like, is this in line with how you think about things? I hope I’m not changing the architecture fundamentally, but we also make it clear internally that like code reviews are quite optional. And, you know, if you’re just changing docs, if you’re just doing, you know, some basic stuff, like don’t wait for a human to look at it, especially in the era of AI, why do you need a human to look at every piece of code? You don’t have to. But that does require you to have, you know, very good controls, you know,

1:04:30 internally around, you know, testing things. Yeah, and exactly. Like, you know, infra companies, like we need one more nine than the average startup, right? In terms of uptime. But like, you know, these are investments you just have to make. Like, you have to build systems that are very resilient to like, you know, you have to have rate limits everywhere. You have to decouple systems. You have to make sure that like, nothing is a single point of failure. They have to have extremely good test coverage. Agents are actually really good at writing tests these days, which was like, you know, a few years ago, I thought like, one of the major things missing in agents, now they’re actually quite good at like, because I think it takes a little bit of like, sort of kind of like, need to step out of what you’re trying to do for a second and be almost a little bit like devil’s advocate to write good tests. But now they’re actually quite good at writing tests too. So I think, I think all these things help. Do you guys use like frontier models for coding or do you guys have you guys experimented with like, open source ones? We experiment with everything. But yeah, I mean, you know, initiative is a lot of cursor, then, you know, of course, Claude and Codex and now a lot of people are using, you know, night shift and all these open source stuff or other stacks. So I don’t necessarily have like a strong point of view, like I want people to experiment and try different things like we, you know, use frontier models, use open source models, we’re trying to do a little bit more dog floating internally. Now that we’re actually running inference for a lot of the open source models, like a very large scale. I think it’s also helpful to like use certain internal endpoints for a lot of agentic development. So we try to promote that a little bit. Something I read maybe like, I don’t know, a year ago, maybe like two years ago,

1:06:00 you were saying that you didn’t think that there was going to be like a big on-prem business for you guys. Have you maybe like changed your opinion on that? No, I have not changed my opinion. I think, moral to me, I’m a Claude maximalist and maybe to take a step back, like when the cloud came in 2008, 2009 or whatever, my first reaction was like, that’s crazy. Like why would I want to take my code and run it on someone else’s computer? That’s like insane. And then like six months later, I was doing it. And I think, you know, like took me six months to change my mind. It took probably enterprises like another 15 to 20 years. But like you look at, you know, vendors like Snowflake or I mean AWS and like, I think there’s many examples of Datadog, et cetera. Like the cloud is clearly winning, right? Like and the multi-tenant systems are clearly winning. And you know, to me, on-prem looks increasingly like sort of a legacy business model. And I actually think, you know, BYOC, you know, sort of VPC peering, like all of these things, you know, like to me, they reflect sort of an outdated security model of like, we need to protect everything inside our own network, right? Like using the network as a mode. And that I don’t think is true anything anymore, right? Like we’ve sort of moved on to more like actually robust security paradigms, like, you know, zero trust and things like that. And so to me, BYOC and those things, like they look increasingly

1:07:30 as like outdated, at least from a security perspective. From our perspective also, like the fact that we have full control over our entire infrastructure stack, we rent all the hardware, we deploy, we manage all the infrastructure, we deploy and like manage health and observe, you know, all the metrics and all the hosts. It means we can iterate very fast and build a greater product. And I think that’s very, very valuable versus, you know, customers in the past who had enterprise editions that they had to deploy on customers environment, they had to be very, you know, they have to think very carefully about release management and having the like, we don’t care. Like if we want to introduce a new database, like we can introduce a new database because we have control over it. Like we don’t need to think about what customers have access to. So to me, multi-tenancy is, is, yeah, has so many different benefits. I mean, that being said, like, you know, if we were to work with like, you know, some very large Fortune 500 customer in the future and, you know, they ask for BYOC, you know, we’ve done this in the past, we’ll give them a number. Like, you know, that number is very high and no one said yes to it so far. But like, who knows? Like, you know, at the right price, like I would still build it. And, and I think there’s like maybe separately, like another thing which I think is interesting, which is like, in theory, BYOC might also be a way to work with and get capacity in certain ways. Like we sometimes work with Neolabs. And those Neolabs have already have very large capacity reservations. And, you know, the question is like, you know, how do we, how do we get them to run modal when they already have capacity? And BYOC might be a solution. We can still keep the control plane in our environment.

1:09:00 And the other flip side of that is like, we could maybe even run like other people’s stuff on that capacity. So there’s some like, kind of fun stuff you could do in theory where like, you can sort of, you know, like kind of aggregate all that capacity into one big pool. So we’re excited about some of those ideas. But in general, like I feel like on-prem, I don’t know if we’re ever going to do. But BYOC, like maybe bring your own cloud. I can say BYOC. I should explain. Let’s say hypothetically, you are going to raise, I should tend more money like soon. What would you use that money for? I mean, like I mentioned, like in this capacity situation, the, a lot of the money, a lot of the cash flow, I should say, that’s required is capacity, prepayments, right? And so that to me represents the best use of cash today. We’re ramping up capacity very aggressively. Maybe we move into, you know, other types of deployment models in the future, ownership or whatever. But I think there’s always going to be a cash component of that, like some sort of down payment or prepayment or some sort of that. So that to me is the greatest use of cash right now. There’s of course other things like, you know, salaries and office spaces and stuff like that. But that actually in the grand scheme of like our financials, like it’s basically a rounding error, right? Yeah, interesting. Do you think that’s going to change? No, I think, you know, the sort of operating leverage that we have in this business is kind of insane. Like, you know, we have 200 employees, you know, they cost a lot of money, but, you know, the

1:10:30 GPUs cost a lot more money. Yeah. If you had to say, like, what are the top, like, I don’t know, like three things that are your biggest focus or the business’s biggest focus right now, like, what would you say those are? Capacity and the sort of financial complexity. And, you know, in general, like, how do we, how do we think about that space? How do we think about pricing and packaging? How do we think about the sort of tension between ownership and renting? Should we be going a more neocloud direction? Should we go in the opposite direction? The sort of counter position, like, that’s probably the biggest thing that I think a lot about. The second biggest thing I think a lot about is probably around, like, more like product strategy. Like, I think a lot about, like, what is the next building blocks? Like, if we, you know, do more like, agentic stuff, like, what are the things we need, you know, are there certain storage primitives or compute primitives? Or in general, like, what is the thing that the world is going to wake up and realize they need over the next few years? And ideally, we have that at that point. We’ve been pretty good in general at, like, you know, anticipating demand and building products in advance of it with the serverless functions in the sandboxes. So what’s the next thing? Then I would say, like, actually the third thing I think a lot about is, like, more on, like, culture and company building and stuff like that. I mean, we recently crossed 200 people. We’re growing extremely fast. We’ll probably hit 250 by the end of the year. Like, when you’re growing very fast, you have 250 people that are all trying to figure out how they can have the most value to modal and, you know, making sure that they have the right context, that they have the right motivation, that they’re excited to come into work every day because they get to build something incredibly cool

1:12:00 and they know, you know, what makes modal successful, what they can do, aligning the incentives and making sure that, you know, we, we, people have like a very positive culture where they have the sort of implicit trust to do a lot of stuff, but also the accountability to like, you know, deliver the best things and, you know, and be held accountable to like, if they’re not doing it, to me, those are like, you know, super critical things around this scale. Like, when you’re like 10, 20, 30 people, you can all just like, kind of sit in a room and like, it’s all kind of implicit and you kind of just like, feel it. People can sort of read, read me like, they can tell me when I get a little bit annoyed. They can tell when I’m happy, but like, now we’re 250 people and like, people don’t necessarily see it and, you know, of course, there’s like a lot of new people that are like, maybe a little bit nervous to like, talk to CEO and like all this stuff. So like, you know, how do I remain approachable and how do I, like, it’s, it’s, there’s like the Dunbar number, which I think is like more sort of, you know, faked than, than anything, like at least like the story behind it. But like, it is actually kind of real still, like there’s something around like 150, 200, we’re like, laying the foundation of the culture is very, very important. How many people are you guys trying to hire over the next year? We’ve grown about four acts on an annualized basis in the last 12 months. I think that’s like an unsustainable growth rate on a relative basis. I’ve seen that, you know, you can double the team every year and that’s not super hard to do, but four acts, what you start to see is, you know, cultural drift, you start to see islands of different working styles.

1:13:30 You start to see people doing like very speculative things like that. That’s something I always watch out for in the engineering team. You go and talk to the engine and engineer and I’m like, what are you working on? And it turns out they’re like building something like kind of like fringe and like, is this really what you should spend time on? So growing very fast has that cost and you know, it has the cost of ramping people up. So I don’t know. I think by the end of the year, maybe 250, about the end of the next year, you know, but like, you know, we’re going to keep growing like maybe two and a half, three acts. So it’s very likely we’ll end up like next year, like 700 people, something like that. I’m going to assume most of those are engineers. Most engineers. We’re wrapping up sales a lot too. Okay. What kinds of engineers? It’s primarily systems engineers. I mean, we’ve always been like very deep in the sort of, you know, Linux and file systems and scalable systems and networking and cloud and stuff like that. So that’s the core of what we’re looking for. Like people are really good at those things. We’re also looking a lot at like, you know, more like, because we moved up in the stack a little bit, like we’re doing a lot of work in the LMA in front of space. So now we’re dealing with a separate set of concerns around, you know, CUDA and like GPUs and and and MFU, you know, like maxing out the tokens that you can get per GPU and a lot of that things. And, you know, even though I actually like will say a lot of the work in that space, in my opinion, turns out is like more systems driven still, because it turns out like KV caching and routing is actually like, is still like very like systems like like these are very like systems have problems. But yeah, a little bit

1:15:00 more like sort of GPU performance, that kind of stuff. We’re starting to build up a research team. We’re very excited about, you know, moving into stuff around like, you know, post training and distillation and better methods for quantization for your customers or for internally. Both. I think it’s like a little bit like with the FDEs. Like I think we initially will do a lot of stuff with our customers to optimize their models. But over time, figure out how do we turn those into products? I think we’ll sort of arguably fail in a certain way. If every time a customer needs to optimize a model, we have to go and work with them. And, you know, long term, it has to be turned into products. And so with the FDEs and with the research team, initially, you’re going to have to do a lot of work with the customers and we are that. But over time, we’re also building a lot of internal tooling and a lot of over time internal products that are fully self-serve. So we’re working a lot of stuff right now in post training space, but also in the GPU performance space. So a lot of cool stuff. A lot of cool stuff. Back to the product strategy stuff that you were talking about. Yeah. Yeah. Like what are your thoughts? Like what do we need over the next year, two years? I think it’s incredibly hard to say. And if I said like, I know exactly what we need to build, I think I would be like delusional. I think what we’ve seen so far, and what I kind of believe in on a more general level is like, we need better building blocks. Like this is a new era of software and we’re unlocking a whole new type of applications that could not be built a few years ago.

1:16:30 Doing all the GNI stuff, code that, LMS that output code that needs to run like in an isolated way. Like there’s so many like new primitives that are needed to work with GPUs and work with LMS. And we built some building blocks so far and that’s unlocked a bunch of applications that the world has seen over the last few years. Not just stuff with us, of course. But I think over the next few years, there’s going to be so many more cool things. And so I mentioned some stuff. Like I’m very bullish on training. I’m very bullish on fast jobs. I’m very bullish on certain things around like agents and low latency and things like that. But like in the end, we consider ourselves like an infrastructure shop, right? Like we’re not trying to build, you know, very narrow things. Like we try to build like very general purpose building blocks and put them in front of customers. And then those customers will combine those building blocks and build some incredible applications that we could have never anticipated. I want to go through some of your tweets. This will be the last part of it. Okay. And we can just like, maybe you want to double click on anything. A lot of long running persistent sandbox stuff right now. Today feels lift and shift. I suspect sandboxes will evolve from dev box into cloud to something more ephemeral, one environment for every command. And now I’m shouting in between so you can pause, resume, clone. Yeah. I mean, we talked about this a lot. Yeah, we talked a bit about it, right? Like I’m fully, I think, I think the median sound box in modal today is like eight seconds long or something like that. Like, which is kind of incredible

1:18:00 when you think about it. And over time, I think it might get even shorter. We want to support long running sandboxes too. Like I think they’re awesome, like very valid use cases around long running sandboxes. But like my vision is like, you know, the world needs also, we need to figure out this like how can we just massively parallelize things and reducing the overhead of starting and stopping sandboxes? I don’t know if I ran an IP already start up engaged in catfax arms race for the competitor, it would be nice to coordinate and paste arms race a bit. No comment. The biggest downside of GPU scarcity is not that it’s hard to get GPUs, but that almost all capacity is sold as three years as a three year rectangle is starting three to six months in the future. That’s not what demand looks like for most users. I think we talked about that a little bit as well. Yeah. And you know, maybe just go a little bit deeper now. Like one thing I think modal has always been very, very good at is fully on demand capacity, right? Like if a customer comes to us and need 100 GPUs, like we can typically give them that like immediately, you don’t even have to talk to salesperson. And a lot of workloads in real life, like they don’t look like these like flat, you know, rectangular workloads, like they scale up and down throughout the 24 cycle, they maybe grow over time, they maybe have certain burst periods. And to me, that is a challenge with a lot of customers today. Like the, and I do think it’s a legit problem for, for the space is like the only, the prime way to get capacities to, to buy these rectangular blocks. It’s not great. I think the world needs to, and that’s fundamentally, I mean, I’m very biased, but like that’s why I think modal can play a very large role in that sort of, you know, whatever you want

1:19:30 to call it, sort of duration transformation of offering more flexible capacity. And we can do that because we can aggregate many, many thousands of customers and the demand gets very smooth and aggregate. So that’s a lot of the value prop of modal is exactly that. Would you say that’s a differentiator? Like the, oh really? Okay. Yeah, massive differentiator. Actually, in fact, like, I think, modals, like if you want to be like very sort of reductive, like you can break down modal’s value prop into two things. What is a developer experience, the ability to like iterate and like build things and not have to think about infrastructure. The other thing is the capacity stuff. The more and more I think about modal and like where we’re going in the long run is like we’re building, like we’re aggregating the world’s compute capacity. And we’re very good at slicing it out like very granularly, because we’ve built all this work around, you know, scaling up and down extremely fast, you know, all this stuff around GPU, snapshotting, working some, some ways to make it much, much faster. And that model works at very large scale, like really, really well, because you can aggregate lots of customers. Every month I keep saying or saying box revenue might apply to it, but it keeps going up 30% every month. It’s kind of nuts. Like I don’t understand, you know, like every month I’m like, well, you know, sandboxes grew a lot, but like clearly the kind of plateau and then they keep growing even more. So I, I, I don’t know. And a lot of VCs, like they’re like, how did you know sandboxes was going to be big? Like, and the truth is like, I didn’t like, like, I thought it was going to be bigger than it was in 2023 for sure. But like the fact that sandboxes is a third of our revenue today, it’s, it’s insane. And, you know, we’re obviously very happy about it. And we have like a very large team investing

1:21:00 in sandboxes. You know, I don’t want to like trivialize it. Like we obviously, I do want to give ourselves like a lot of credit for, for seeing sandboxes for what it is today a little bit back in even 2023. But I think, you know, if like, I probably would have never, you know, taken a bet that revenue would be this large from sandboxes and the market demand would be so much. There’s an interesting structural reason why inference providers should exist separately from model builders. The best model keep changing a lot inference providers keep can easily reallocate resources between models since overall demand is quite smooth. So I mean, if you think about all the Chinese labs and, you know, in the future, hopefully more US labs too, every time a new lab releases, you know, the latest and greatest open source models, like you see big shifts in demand, you know, for these models. And so I think a challenge with, you know, if all these labs independently like held capacity, they would see when they launched a new model demand would go up a lot. And then another lab launches a better model and then the demand goes down. You know, but, but of course, like in the background, they’re working on the next version. So like, I think, you know, there’s like a structural reason, like I said, like why you’re going to start, you’re seeing demand kind of shifting to like the latest and greatest Chinese lab. I mean, not like, it’s not like, you know, this is like, everything just moves overnight because, because a lot of models and prompts and stuff like, like a lot of things that get sort of there’s like a certain stickiness to it. But I do think, you know, there’s an argument that it’s very easy for model to like, you know, right, today we’re running, you know, X number of 1000 GPUs for one model. Tomorrow we need to shift like, you know, 50% of that to another model.

1:22:30 Sure, that’s just like, that’s what the system is built to handle. No one even needs to tell us to do that. It just magically does that. So you agree that basically the LLMs are commodities? And if you agree, then where do you think the business for the big labs is going to be? I’m not sure if that’s necessarily true, right? Like, is Taylor Swift the commodities? Like, she releases a big album and goes on a tour and makes a lot of money. And then, you know, then there’s like a down the air when she goes and like records, I don’t know, this is best analogy. And then she releases a new album, you know, so I don’t know, like she still has like a lot of brand and there’s a lot of, you know, power and like makes a lot of margin. So like, as long as the labs like repeatedly can launch new models, and as long as the margin that they have on those models are such that they can sort of pay off amortized like that the cost of training those models, it could still be like a decent business, I think. So you don’t think that the intelligence is like plateauing? No, I don’t think so. I mean, I think it remains to be seen, right? Like, models are today are like very good at like doing things that can be trained very efficiently, you know, using reinforcement learning. So we’ve seen that initially with, you know, coding and writing simple snippets over time, it’s gotten really good at also writing production like level code. It’s gotten really good at math. It’s gotten really good at cybersecurity. Like these are all like very quite verifiable things. I still think it remains to be seen, you know, our model is going to be really good at whatever, cooking food or I don’t know, like things that I have like more

1:24:00 like diffuse targets, maybe longer feedback loops or whatever, I think humans probably like my feeling is like will be quite good and quite competitive with models for a long time in those domains. But you know, certainly, you know, AI models, they’re getting very good as I’m sure everyone can agree with. Yeah, so obviously, like the models are getting really good at coding, they’re already really good at coding. What other applications do you think maybe that aren’t as big right now are going to become huge over the next few years? For the LLMs? I don’t know, but I think anything that has like kind of a verifiable computational bio, I think you mentioned. Yeah, computational bio, I think those are like, not exactly verifiable, but there’s like certain ways you can sort of run simulate, like you can use an LLM to output predictions, then you can do like computational like, like methods to like simulate molecular dynamics and get like kind of feedback loops. I mean, like I think like material science is like maybe another area where like you have a little bit of that loop where you can sort of, you know, synthetically manufacture certain materials and measure the properties and use kind of put an agent in the loop and like, you know, do things. So, you know, I have a physics background, I’m pretty excited about that. So drugs, materials, what else? I’m drawing a blank, but I’m sure there’s many other things. Cool. I mean, those are all the questions that I had. Is there anything you wanted to talk about that I haven’t? I’m good. This is great. This is a lot of fun. Awesome. Thank you.