it looks like the core concept of “derived data” was heavily inspired by Mononoke, the server component of Sapling
I want to be clear that it isn’t just the concept of derived data that is inspired by Mononoke. A Scion changeset is almost exactly the same as what we in Mononoke called a Bonsai changeset. I led the design and initial implementation work for Bonsai changesets back in 2018, including writing the four canonical design documents that are still the reference. The advantages of Bonsai changesets are very non-obvious until you spend a great deal of time thinking about the problem, which I had given my years of experience on version control.
This is not surprising given the additional information I received later, which includes the fact that Claude was pointed directly at Mononoke and told to copy it. The fact that none of this is even remotely hinted at is shameful; taking an existing battle-tested (over a decade!) design and copying it in 5 months is much less impressive than coming up with a brand new design and productionizing it in 5 months.
edit to add that there are also other parts such as an API to do source control queries and mutations with that are copied from Mononoke. Those have been done elsewhere before Mononoke, too. In general it is always good to situate your work in the larger context of what came before you.
I did say in the comments to Anthropic post, that they don’t say enough to figure out what they took and combined from known and tried ideas.
I would say that stressing derived data (as in your main pot) does not really feel like much more convincing, as derived data at the level they describe it is just database indexing and caching, so who knows.
But the claim you make here in the comment, this one is a damnable omission on their part. What exactly goes into the changeset in Bonsai is clearly a careful optimisation for a monorepo use case (my dream DVCS for personal use would have functionality incompatible with that!), and when they closely followed that, it’s a clearly more of a fork/rewrite than «hard-won lessons», and yeah, that needed to be explicitly credited to Mononoke and Bonsai changesets.
Thanks a lot, that was much more substantial to read than the first version!
Re: contrarian — as a happy Monotone (a completely different and much older Mono?o?e VCS!) user and someone who packages enough stuff to have been forced to use Darcs slightly outside the happy path, I would say that contrarianism of that approach does not impress me as contrarian, but figuring out exact details of the core design to mait the throughput goals possible is indeed a great achievement.
I wonder who will be the first to break, swallow the eventual consistence pill, and remember that both Monotone and Mercurial track branches in a way that allows multiple heads, which avoids full-linearisation requirement. I prefer Monotone’s «global branch names, so you cannot avoid multiple heads» approach for small-scale workflow reasons too, but it also hits the nail on the head for a slightly catastrophic scaling-out scenario.
FWIW, I think server-side rebases with the Bonsai model allow you to have a total linear order on main very efficiently. This is what we called “push rebase” – the original version was done within Mercurial, using files on disk and the Mercurial data model. Mononoke’s contribution here was to implement push rebases efficiently using the Bonsai model, in the context of a stateless server built on top of an object store. As you’ve been able to tell, Bonsai changesets and efficient push rebases were carefully designed to work together and scale up (they’ve scaled up just fine to an order of magnitude beyond the original design parameters).
Linearity still forces one to have a mutex for updating the head position, no? Although with good design it can scale up very well (out, that’s harder)
(Also rebases lie about what was tested, but fortunately nobody ever reads history to see those lies)
Yes. The goal of the Bonsai data model is to make that critical section as small, and involve as few pointer chases, as possible.
Agreed that push rebases don’t detect semantic merge conflicts across files (they worked at the file and directory level). In massive monorepos this isn’t a major problem though, especially because work would generally be done against the latest “warm” bookmark, which meant a bookmark for which the build system had already verified things and cached build results.
I led the design and initial implementation work for Bonsai changesets back in 2018, including writing the four canonical design documents that are still the reference. The advantages of Bonsai changesets are very non-obvious until you spend a great deal of time thinking about the problem, which I had given my years of experience on version control.
I work in public research, and I recognize this situation as “you should write a paper about this”. The description that you use in your post comes from, if I understand correctly, personal documents you wrote for a job application. Curiosity question: have you / your team considered writing actual papers / public technical reports about these things? (Why not?)
Thanks.
FYI in your post you say:
I want to be clear that it isn’t just the concept of derived data that is inspired by Mononoke. A Scion changeset is almost exactly the same as what we in Mononoke called a Bonsai changeset. I led the design and initial implementation work for Bonsai changesets back in 2018, including writing the four canonical design documents that are still the reference. The advantages of Bonsai changesets are very non-obvious until you spend a great deal of time thinking about the problem, which I had given my years of experience on version control.
This is not surprising given the additional information I received later, which includes the fact that Claude was pointed directly at Mononoke and told to copy it. The fact that none of this is even remotely hinted at is shameful; taking an existing battle-tested (over a decade!) design and copying it in 5 months is much less impressive than coming up with a brand new design and productionizing it in 5 months.
edit to add that there are also other parts such as an API to do source control queries and mutations with that are copied from Mononoke. Those have been done elsewhere before Mononoke, too. In general it is always good to situate your work in the larger context of what came before you.
score 3 · canonical
I did say in the comments to Anthropic post, that they don’t say enough to figure out what they took and combined from known and tried ideas.
I would say that stressing derived data (as in your main pot) does not really feel like much more convincing, as derived data at the level they describe it is just database indexing and caching, so who knows.
But the claim you make here in the comment, this one is a damnable omission on their part. What exactly goes into the changeset in Bonsai is clearly a careful optimisation for a monorepo use case (my dream DVCS for personal use would have functionality incompatible with that!), and when they closely followed that, it’s a clearly more of a fork/rewrite than «hard-won lessons», and yeah, that needed to be explicitly credited to Mononoke and Bonsai changesets.
score 1 · canonical
Thanks. I’ve expanded on that in a followup. The original was written while I was upset, and was not my clearest technical writing.
score 1 · canonical
Thanks a lot, that was much more substantial to read than the first version!
Re: contrarian — as a happy Monotone (a completely different and much older Mono?o?e VCS!) user and someone who packages enough stuff to have been forced to use Darcs slightly outside the happy path, I would say that contrarianism of that approach does not impress me as contrarian, but figuring out exact details of the core design to mait the throughput goals possible is indeed a great achievement.
score 1 · canonical
The resemblance of the name Mononoke to Monotone is not a coincidence!
score 1 · canonical
I wonder who will be the first to break, swallow the eventual consistence pill, and remember that both Monotone and Mercurial track branches in a way that allows multiple heads, which avoids full-linearisation requirement. I prefer Monotone’s «global branch names, so you cannot avoid multiple heads» approach for small-scale workflow reasons too, but it also hits the nail on the head for a slightly catastrophic scaling-out scenario.
score 1 · canonical
FWIW, I think server-side rebases with the Bonsai model allow you to have a total linear order on main very efficiently. This is what we called “push rebase” – the original version was done within Mercurial, using files on disk and the Mercurial data model. Mononoke’s contribution here was to implement push rebases efficiently using the Bonsai model, in the context of a stateless server built on top of an object store. As you’ve been able to tell, Bonsai changesets and efficient push rebases were carefully designed to work together and scale up (they’ve scaled up just fine to an order of magnitude beyond the original design parameters).
score 1 · canonical
Linearity still forces one to have a mutex for updating the head position, no? Although with good design it can scale up very well (out, that’s harder)
(Also rebases lie about what was tested, but fortunately nobody ever reads history to see those lies)
score 1 · canonical
Yes. The goal of the Bonsai data model is to make that critical section as small, and involve as few pointer chases, as possible.
Agreed that push rebases don’t detect semantic merge conflicts across files (they worked at the file and directory level). In massive monorepos this isn’t a major problem though, especially because work would generally be done against the latest “warm” bookmark, which meant a bookmark for which the build system had already verified things and cached build results.
score 1 · canonical
I work in public research, and I recognize this situation as “you should write a paper about this”. The description that you use in your post comes from, if I understand correctly, personal documents you wrote for a job application. Curiosity question: have you / your team considered writing actual papers / public technical reports about these things? (Why not?)
score 1 · canonical
Let’s just say I wasn’t as writing-pilled back then as I am now, especially after I joined Oxide.
score 1 · canonical
I’d go further and say it outright shows the touted efficiency gains of LLMs are complete bullshit.
I hope their vibe coded garbage VCS malfunctions and deletes some important code.
score 2 · canonical