lobsters score: 3 · tags: ai
The optimist in me thinks anyone who retains skills will make bank cleaning up.
The pessimist (realist?) in me thinks that society will just decide that software is bad and unmaintainable and that’s just how it is, so we’ll all spend our common resources on mechanisms (including more AI, sigh) to live with shit. I.e. enshittification turned up to 11. Or eleven hundred.
score 2 · canonical
I may be even more of a pessimist and say that we’ve already been heading in that unmaintainable direction for a long time, and AI is simply accelerating it. The core reason behind that is the current economic structure and all of the incentives toward profit seeking over building solid, maintainable, extensible tools and systems.
But I think there’s hope. We’re in a time of great uncertainty and change, and that presents opportunity to shift the narrative and the broad consensus about whether or not that is inevitable. My next few essays will be exploring this more in depth, and proposing an alternative model based on the learnings from the OSS system that already powers Silicon Valley.
score 1 · canonical
I do think long term there is hope. However, whether we can remain solvent while the bubble remain irrational remains to be seen.
Another theory of mine is there’s always been a delta between quality and slop even before AI. But, it was rarely enough for the market to care. But, maybe the delta will become so large, and the skillset to create quality so rare, that quality can win the market again?
score 1 · canonical
It’s a market for lemons — hard to evaluate reliability in advance. Normally, this gets resolved via regulation (possibly it’s just the liability rules that get an update). Typically, it comes after a large enough mess that not regulating becomes untenable…
score 1 · canonical
Funnily enough I have a coworker that has been exploring how to keep it at bay, I’ve been bouncing a lot of ideas with him and exploring the space. He’s building something called coherence.
It’s mostly explorative and it’s absolutely not ready for any type of consumption beyond some internal testing, but so far it works fairly well.
score 1 · canonical
I have to say, I’m very interested in the concept of decoherence as a way to explain the reasons why I see AI doesn’t really work for creating new code in a way that results in “good enough” code. Please note that I’m putting aside moral considerations to adapt to the style of the discussions present in these forums, but I do have moral concerns with the way that LLMs generate code. Namely that it seems to grab code verbatim from open source code and claim it as its original creation without providing the original author its credit. And other ethical considerations around workers rights, and of course the climate impacts.
But beyond that, LLMs do not produce code that is good enough in cases where collaboration is required. I’m sure at this point we’ve all seen cases of someone who is a “AI-superstar” excelling at all of the metrics but leading failing projects, because no matter how fast code can be created, the shared understanding we have around the code that we are creating, and why we are writing the code are still important factors. Does it matter if you write code fast, but no-one in your team can understand it and maintain it once you are gone? Does it matter if you write code fast, but the code does not meet the requirements of the user?
In my view, the answer to both of the above is no: having software that aligns to the need of the users is the most important, and software is not useful if it cannot be modified and maintained as needs change. This aligns pretty well with your concept of decoherence. Never mind the impact for organisations larger than a single team.
score 2 · canonical
Very good writing, and I agree with the analysis. I might disagree that this is a bad thing, though. With these tools and the impossibility of having a globally coherent software stack clearly there will be problems, but there are also positive outcomes, meaning more autonomy at the local level because their corner of the maze is inscrutable. For the Engineer’s mind this is a net negative, but these tools will enable non-technical people to produce “broken by default” software that serves their purpose without having to cohere with a bigger system as it is now. This means, in many ways, undermining Big Tech and creating more technological autonomy for local communities. Obviously the Engineer is not concerned with this stuff, but it’s part of the picture: the monopoly of the Engineer might be over.
score 1 · canonical
Hmm. So, funnily enough, this is close to where I’ll end up near the end of my essays, in terms of where I think tech should be going. But the thing is, I think to get there we actually need to push more deeply into better abstractions and a more coherent ecosystem.
I discuss this a bit in my AI usage policy in the section on code. Specifically, the idea behind a universal design system, which would enable much more flexible UIs driven by agentic development, but on much stricter guard rails that would prevent decoherence from seeping into the core layers and leave it just on the surface.
score 1 · canonical
Genuinely interesting article. I have just one remark:
resulting in runaway exponential growth
I’m always wary when posts throw around “exponential growth”, or expressions like “99% of times”.
Is it meant literally, or is it just a figure of speech. The latter would be disappointing for an article with a scientific slant, especially since it repeats it 4 times.
I ask because, honestly, I’m not convinced that the source of decoherence described in the post could produce anything faster than polynomial growth.
score 1 · canonical
The exponential growth part is meant literally, I genuinely believe the growth in complexity of truly decoherent systems in combinatorial and exponential. It can be extremely bad!
The biggest issue with proving this in a meaningful sense is that it would rely on a lot of counterfactuals. The best example that I am familiar with is probably various layers and functions of US government bureaucracy. You can find many descriptions and examples of how layered and complex government systems are (see for instance Recoding America), and I would categorize this as decoherence. My sense is that if we made these systems more coherent, then they would also be much more efficient and much easier to change. But we can’t know for a fact that this would be true, because it would essentially be an alternate history.
On the other side though, I think we can compare the store of Windows vs Linux. At one point, Windows Server was the dominant server implementation and growing. Linux was the underdog and it was not treated seriously, with significantly less funding put toward it. Yet in the last 20 years, Linux has become absolutely dominant by count of machines spread across the world, and Windows really only maintains its foothold on the desktop.
Now, imagine if Linux hadn’t been successful, and specifically, imagine building Docker exclusively for Windows. Imagine having to manage not only the complexity of spinning up and tearing down Windows servers dynamically, but also the complexity of the licensing, the various bugs and issues you could encounter, and being dependent on Microsoft entirely for fixes since it would all be closed source. That would, IMO, be a decoherent mess, and the economic impact would likely have been far fewer successful startups, many orders of magnitude less economic activity, and a much less vibrant web.
score 1 · canonical