跳到正文
Hacker News· sunshowers·· 3 天前AI 评分40

Anthropic 的 Scion 版本控制系统被指大量借鉴 Meta 的 Mononoke

Anthropic's Scion is heavily inspired by Mononoke

AI 导读

Anthropic 新发布的版本控制系统 Scion 使用"derived data"概念优化 blame 等操作,被指与 Meta 的 Mononoke 源码控制服务器高度相似。

正文 · AI 翻译

在 Hacker News 或 Lobsters 上讨论。

这里是 Anthropic 昨天描述(存档链接)其新版本控制系统 Scion 如何使用一种称为“派生数据”的概念,使 blame 等操作更高效:

A changeset records only what’s changed, so on its own, it can’t quickly answer questions like who last edited a line or which commits affected a file. To answer those, Scion builds additional data structures from the changesets, including dedicated structures for file blame and path history and skip lists for the commit graph. We call everything that isn’t directly part of the changeset “derived data,” and we compute it all through one generic mechanism. Each type of derived data is defined recursively, so its value for a changeset is computed from its values for that changeset’s parents. This mechanism has three benefits. First, we can easily add a new type of derived data when a new data structure would make some operation faster. Second, for each type, we can choose whether to compute it eagerly or defer the work, so we avoid materializing data we don’t need. And third, because the client never sees the internals of these data structures, we can optimize them without changing the client. We’ve added new types of derived data many times as the system has grown, such as new indices for performing copy detection or path-based file search.
Anthropic 如何描述 Scion 的派生数据。

这是我在 2022 年,在我的 Oxide 候选材料中描述我帮助构建的 Mononoke 源代码控制服务器如何使用一种称为“派生数据”的概念,使 blame 等操作更高效:

Derived data. While Bonsai changesets work great for pushes, they work less well for things like diffs. Git diff commit1..commit2 is an O(n log n) operation where n is the number of changed files – with Bonsai changesets, you’d have to look at every single commit between commit1 and commit2 to compute the list of files changed. For a large repository this can be orders of magnitude slower (and compounds the pointer-chasing problem). To solve this, we moved the diff-friendly representation to being derived data (i.e. an index). The server creates a separate Merkle DAG with a Git-like tree/blob model, with pointers back to Bonsai changesets as required. Note that: Derived data can be computed and stored asynchronously, outside of the main critical section. The pointers between commits and trees are flipped around. With the Git data model, the text of a commit contains the hash of the root tree. With the Bonsai model, the text of the root tree contains the hash of the commit that introduced it. This gave us the ability to build out other derived indexes for whatever other operations we wanted. For example, git blame is quite slow on large git repositories, requiring a traversal of the entire repository (not just commits that touch the file). With Mononoke, we built out a blame-friendly index where each version of each file was a separate node in the Merkle DAG, with its parent set to the previous version of the file, and with pointers back to Bonsai changesets. This meant that blames became O(number of commits that touch the file) rather than O(number of commits in the repository). Over time, the team has added more indexes to handle specific use cases, and Mononoke now has 8 separate indexes. Adding a new index is a relatively cheap operation that doesn’t involve any fundamental changes to data structures.
我如何描述 Mononoke 的派生数据。

我认为这种模仿程度远远超出了拥有相似见解1。Sapling 只被归功于“来之不易的经验教训”,而 Mononoke 则完全未被提及,这近乎剽窃。这篇文章给人的印象是 Anthropic 声称 Claude 设计了这一切,而现实是我们在 Meta 的(非凡的)团队提出了许多见解,Claude 在近十年后才忠实地应用了这些见解。(我们也研究了 Mononoke 和 Sapling 的大量先前技术,从 Google 的 SrcFS 到数据库中的预写日志。)

我非常高兴我们团队在 2017-18 年的愿景如此强大且持久——当你逆流而行时,你总是不太确定自己所做的是天才之举还是疯狂之举,而我认为 Mononoke 的设计经受住了时间的考验。而且,明确地说,我并不反对 Anthropic 复制 Mononoke 的设计。(这也是我们开源的部分原因——通过传播好想法来造福人类。)

但随着 LLM 继续协助设计新系统,它们很可能会严重依赖现有设计。Mononoke 是开源的,并且在每个 LLM 的训练集中。如果你问 Opus 5.5 或 GPT-6 Astra(感谢 Matthew Fowle)关于先前技术,它们都会立即指出与 Mononoke 的相似之处。

向先辈致敬现在比以往任何时候都更容易——也更重要。不再有借口了。


  1. 关于一个具有相似见解但许多关键设计不同的系统示例,请参阅 Vicent Marti 为 Cursor 撰写的博客文章。 ↩︎

来源:Hacker News · sunshowers.io