lobsters score: 3 · tags: programming
I note that none of the “reasons to dislike” relate to the real, documented cognitive atrophy seen by consistent users of the models as they surrender more of their thought processes and self-confidence to the machine.
Perhaps because the author doesn’t have a succinct refutation for that?
score 2 · canonical
« Here are all the reasons I found an argument against. Let’s pretend other reasons do not exist »
- AI relies on stolen intellectual property
- AI accelerates the planet resources exhaustion
- AI atrophies education and critical thinking
- Ai will extinguish humanity in 10 years
- …
score 1 · canonical
This post is basically “it’s always been this way” four times in a row, as if that means we’re not allowed to complain. As if everyone was happy with the state of software, or the world, beforehand. As if highlighting problems afresh is a reasonable excuse to accelerate them. At what point is this kind of post just capitalist propaganda?
score 1 · canonical
Funnily enough, it’s coming from a self-described (albeit top-hat-wearing) communist.
Boy, that temptation of ~personal slaves~ offloading hard work sure is something, huh.
score 1 · canonical
At what point is this kind of post just capitalist propaganda?
It’s not, it’s example #387946 of “Your specific X/Y/Z grievance is actually a general anti-capitalist grievance”.
Socialists tend to say variants of this a lot because many people never connect the dots between their specific complaint, and the system that caused it. It’s an attempt to get people to focus on the disease, and not the particular symptom du jour.
Many, but not all, anti-AI complaints fall into this category.
score 1 · canonical
“Your specific X/Y/Z grievance is actually a general anti-capitalist grievance”
Which is supremely unhelpful, vein just a “oh you’ve only just noticed there are promlems? well I, the better, more smart person, have known about problems for a while”. Instead of being condescending, why not talk about collective bargaining? Why not shout out some software-related unions?
score 1 · canonical
You can certainly take it that way, but my observation is it’s still an uphill battle to get people to even see the problems are related, so I prefer to focus on that first.
I actually think suggesting devs engage in collective bargaining or form a union is an even bigger leap, too big for first steps.
score 1 · canonical
It’s an attempt to get people to focus on the disease, and not the particular symptom du jour.
When the symptom is the focus of the author’s consultancy I fear this piece may not be as anti-capitalist as you interpret. If the post is indeed about focusing on capitalism as a whole, if anything it reads more like an attempt to shift focus away from the thing people can potentially make a difference on right now.
score 1 · canonical
It’s an odd duck because the author also recently wrote what I consider a very good journal article: https://deprogrammaticaipsum.com/fully-automated-luxury-communistical-software-factories/
score 1 · canonical
I don’t like AI coding, but I can’t say it’s not useful. Most of the arguments in this article can be disarmed by taking the abstraction “one step up”. LLM generated modules and software, microservices, whatever your level of abstraction… now need to be composited together. You go and deal with higher order problems, factories, generation, etc.
Is it a way to develop critical bits of code, depths of your business logic? Absolutely not. Can it solve a lot of boilerplate and support logic? Absolutely yes.
Seeing “AI Companies as Bad Actors”… can’t fully disagree there. There’s far too many far too dodgy things about these businesses, and I trust them as far as I can audit them (not very far). And the missing argument’s the cognitive decline of using these things to think for us. I try to keep myself sharp by still manually doing a fair amount of things in my life. I refuse to automate everything - especially the stuff I do for fun. Having an LLM manage hobby-adjacent stuff is a quick trip to burnout land, especially once fixing the thing becomes “the hobby”… and it does.
score 1 · canonical
I see agents as more like a cnc and its effect on manufacturing. You can still make things by hand but for the most part you can achieve the highest quality and scale with automation. Obviously as we become cnc programmers and operators we lose the hand-filing skills we once used, just as our forefathers forgot how to use flint-knapped hand axes.
I think a big part of the angst is that we’ve had to look around and engage with what the things we’re building are actually for, instead of just resting in our craft. There’s lots of good work to be done for things that are good. The question is how do we (or our agent wielding successors) only work on good stuff and not bad stuff.
score 2 · canonical
I’d say the closer analogy to CNC is the compiler. The input to CNC is still a precise transformation of the intent of the designer. It does the work faster but it’s the same work. On the other hand, sometimes a human needs to precisely define the lower level, doing some manual filing because the cutting tool won’t fit in the space, or writing some assembly because the compiler doesn’t know how to do indirect threaded code.
As a craftsperson or client you may look at the result and miss the imperfections (the “hand”, it’s called) of the old way: the curvature is implausibly perfect, or the assembly is too clever to have been written by a human. But you can always see the human intent everywhere.
LLMs, on the other hand, take imprecise descriptions and turn them into precise output by making stuff up instead of waiting for the human to fill the gap. It’s the “imprecise description” part that’s really new. I don’t know of a very good analogy for that in any pre-existing tool.
score 1 · canonical
I see the “imprecise description” as a spectrum;
Somewhere there is an optimizing compiler, where you give it intended semantics and it “makes stuff up” for the codegen that satisfies those semantics, hopefully in a way more optimal than the original description translated verbatim. Then somewhere far else on that spectrum is an LLM which does the same thing, but with general purpose tasks, not just codegen for a specified semantic operation set (can do it deterministically as well).
There’s always been “imprecise translators”. How people use them effectively is by building up and intuition or mental model about how that translator processes their input to help generate the desired output, even if they can’t trace by hand what exact decisions it’s making.
What would be the new thing in this case (genuine question)? Would it be the “codegen targets” of the translator, it’s “acceptable frontend IRs”, some other capability entirely?
score 1 · canonical
I agree it’s a spectrum, but at some point just saying “it’s a spectrum” erases the interesting distinctions. The boundary of impreciseness that can’t be crossed by the “compiler” used to be limited to a fairly narrow range. Think of the possible abstracts of a paper in a PLT journal.
But with an LLM you can literally say nothing but “make a website for scheduling a kids’ soccer league” and it will do it. That is, even if I had a mental model, I didn’t tell the “compiler” what it was. Not only that, while it’s working the “compiler” can ask me what my mental model is, rather than just telling me the model I gave it is incomplete or inconsistent and stopping.
And where it really gets into a new category for me is that the output of the compiler isn’t even defined. You could just as well say “make a presentation for a soccer league” or “get the accounting in order for my soccer league” or “automatically answer questions about our soccer league”. The analogy just seems to completely fall apart at that point.
score 1 · canonical
“Mental model” is an expectation of how the tool works / will translate or react to some given input. It’s not something you tell the tool - it’s how you reason about what to tell it. We do this for human personalities too.
In this case, the mental model I have for SOTA models when it comes to “make a website” is that it’ll write some ugly js & HTML but it can get it done. But the mental model for a older smaller model is that ill need to give it a lot of guidance to achieve that.
For the case of “LLM can ask clarifying questions”, this is a harness diff. When you type statically detected UB or compile errors in the compiler example, it technically could interactively ask you want you mean for each instance. The rust compiler does half of this as well. It’s just usually inefficient when “IDE is a chat interface” isn’t true.
The output of LLMs is defined as much as optimizing compilers is: you give some code and it has infinite ways to achieve some set of behavior. Maybe I’m misunderstanding what you mean by defined?
score 1 · canonical
Sorry, I misread your earlier comment. In my reply when I said “mental model” I meant your mental model of the thing you’re trying to build, not of the tool.
The output of LLMs is defined as much as optimizing compilers is
It definitely is not. Compilers perform a very well defined transformation from input to behavior (not 100% defined, but as close as practical politics can get it). A compiler that “optimizes” by violating that definition is a broken compiler. (The 1% that’s undefined causes a lot of discussion, which takes place while everyone relies absolutely on the other 99% being correct.)
Whereas in most cases you can’t even tell if the output of an LLM did achieve desired behavior, because the input isn’t precise enough to determine the desired behavior in the first place. LLMs not backstopped by deterministic tools (compilers, test harnesses, proof engines, even just plain reliable arithmetic) in a loop have a hard time even making artifacts that are functional.
score 1 · canonical
Humans have real issues writing large chunks of C++ without UBs, though. Looking at what each GCC bump uncovers across a large distro, I do not really believe compilers being used as well-defined transformations within their well-defined zone is a reality. And all fast-moving compilers have a long list of actually-correct code they miscompile, it is just about exotic enough combinations that people do not prioritise cleaning this all up.
It’s the same tolerance to for-profit companies releasing paid-for network-touching software full of UBs that now enables careless use of LLM-generated code is well. I sometimes wonder if the correct approach would be to pretend to believe, and to say that now that LLMs can fix all the bugs any defects are intentionally left in thus normal consumer product liability rules can finally be applied to proprietary software by for-profit companies.
Speaking of notionally-functional, I am kind of surprised of what latest-gen local-ish LLMs (aggressively quantised 120B or less aggressively quantised smaller ones) sometimes can one-shot. And if you do not give me a compiler/interpreter as a deterministic backstop, neither am I any good at one-shotting longer pieces of code than that.
score 1 · canonical
LLMs not backstopped by deterministic tools […] in a loop have a hard time even making artifacts that are functional.
This isnt the case for most frontier models today; They can do the job of an optimizing compiler (generate working & efficient) asm from just source without calling out to verification tools. Even with pseudocode unseen before by guestimating semantics.
I agree though that it’s not the case for smaller models. Mixing both under the “LLM” umbrella, while technically correct, loses a lot of nuance for claims.
the input isn’t precise enough to determine the desired behavior in the first place.
The evaluator/prompter can. It’s how miscompilations and misoptimizations are found in existing compilers.
Compilers perform a very well defined transformation from input to behavior.
The behavior is defined. The transformation is not.
If you have a set of bitwise and ternary expressions with bit operations, theres hundreds of ways to get that behavior; Could it
vpternlogwith avblendmpat the end for multiple instances? Could it cmp+branch for each one? Could it eager-eval thencmov? These decisions and their various heuristics change across compiler versions.Only a few years ago, you had to use inline asm to get clang to emit
lock btson x86. Now, it understands the semantics ofatomic_fetch_or_*, the args going into it, and how its result is operated on, such that you can write the semantically equiv without hacks to getlock btscodegen. But if it can’t figure some aspects out, it still falls back to alock cmpxchgloop. The transformations (or “mental model”) is something intuitively built up as you use the specific tool, not defined somewhere. This applies to LLMs as well.
score 1 · canonical
Not passing comment on the analogy (unless…?) but the categories of machining work it makes sense to be CNC-only are not that broad. It’s just that there’s a lot of times you’d want 50-10,000 identical widgets. There’s always going to be a reason to pick up a file – and often that’s the best way to get the highest quality work. :)
score 1 · canonical
Huh, https://www.destroyallsoftware.com/talks/useing-youre-types-good is gone. Would have liked to see that. Post itself seems kind of light on content.
score 1 · canonical
deleted by creator