Humans are bad at writing secure code, and GNOME developers are no exception. GNOME is primarily written using unsafe programming languages where simple mistakes in our code lead to devastating consequences for our users, and we make these mistakes all the time. No matter how much we try, GNOME developers will fail write secure code...
Ah, yes, ignoring all of the negative externalities (like pollution), massive capital investment in data centers, LLM companies not turning a profit, etc. we can pretend the slop machines are really cheap!
If a drug dealer offers me the first hit for free, that’s free drugs, that’s fantastic, I love free drugs.
Isn’t it pretty clear though that for something like a security scan or similar that even the externalities included price far cheaper than the alternative? Whether that’s a good thing or not can still be an open question but I’d be interested in hearing the argument that LLMs are only cost effective for finding vulnerabilities because tokens are subsidized/should cost more for some other reason
I’m not disputing that the prices as is may not reflect the true cost of the tokens, or anything like that. What I’m trying to get at is that I think it’s clear that LLMs are extremely good at this sort of thing, so even if it were true that the price of the tokens were 100x off, for this particular use case it still might be viable given the counterfactual is something like “spend a lot of money paying pentesters”.
I agree with you that 1M on pentester salaries is likely to produce a better result than 100 dollars to whatever your favorite LLM provider is, but what I’d be interested to know is what you think the breakpoint there is and how far off current token prices are from their true cost
When you compare emissions from a computer to emissions from a human, this is a false comparison.
The human already exists and will be producing more or less the same emissions no matter what work you assign to them. However many pentesters you hire, if they are working on pentesting instead of e.g. web development, there’s negligible difference in emissions. Even if they are unemployed and sitting around at home, they will produce similar emissions, perhaps slightly less if they lose their work commute and/or videoconferencing.
The only way you get rid of the human emissions is if you kill the human. The only way to drastically reduce a human’s emissions is to drastically change their society, e.g. by destroying all fossil fuel generators, or by deporting them to an undeveloped country with much lower emissions (changing which society they exist in). Do either poorly and it’s just a fancy way to kill the human.
But you could turn off the LLM data centers and shut down their gas turbines, without killing sentient beings, and their emissions would stop. Humans survived without LLMs only a few years ago, while fossil fuel infrastructure has been baked into our society for over a century.
It feels like you didn’t really read or understand what I said. $100 in tokens is not the true cost. Let me rephrase:
Negative externalities by definition are not included in the price, and pollution is only one of many negative externalities from LLMs. I’ll mention scrapers pushing websites offline or behind paywalls, and RAM prices making personal computing less accessible, but I could spend days listing harms not captured in the price, and then you wouldn’t read my post because it would be too long.
Capital costs may be better considered of as embodied emissions in this context. The emissions from running an LLM doesn’t reflect the emissions that were created while building the building, manufacturing the computers inside it, or training the model. It’s not correct to only count the incremental emissions from additional use of the LLM.
Losing money means that LLM companies are somehow failing to make back their capital costs. It may be a play to dominate the market and therefore be in a position to raise prices later, it may be incompetence, but whatever the cause, it’s financially unsustainable. It seems the prices of tokens don’t even reflect the true cost to the company, let alone the true costs to society.
So, salaries for human pentesters cannot be compared to tokens for slop machines.
I also don’t think you can get the same results as $1 million on pentester salaries with $100 in tokens, but whatever.
I also don’t think you can get the same results as $1 million on pentester salaries with $100 in tokens, but whatever.
You will certainly get an equivalent security analysis for orders of magnitude less than $1m in tokens though.
Anyway, your alternative scenarios for humans (pentest or loaf at home) leave out the obvious: move humans away from activity where computers do it better, and have them do something more productive, like high-level architecture and design. It’s the same logic as switching to power drills and cranes, instead of making artisanal woodworking.
Your focus on the costs without mentioning the benefits, assumes LLMs are useless (given you call them “slop machines”). But since they have workable use cases (and in the case of security, they clearly do), then it becomes like any other tech, where we weigh tradeoffs.
I personally think the environmental harms will be short-term, and mitigated over time in a variety of ways. More efficient models, cheap software rewrites into more efficient langs, increased model efficiency from more software, model-driven research into environmental tech, etc.
(Now, whether humanity would actually build the current amount of stuff more efficiently, or just build more is a separate question. But that’s much older than LLMs, and is more your-underlying-problem-is-actually-capitalism again, which cannot be solved by addressing the symptoms instead of the root cause. But that’s a whole 'nother conversation.)
There is some very strange cognitive dissonance in your comment. On the one hand, you say that efficiency will solve the climate impact of LLMs. On the other hand, you recognize that efficiency won’t solve the climate impact, in the same comment! You handwave the problem with efficiency away by blaming it on capitalism, but who cares why efficiency doesn’t work if you’ve just invalidated your own solution?
For those not familiar, we’re discussing the Jevons Paradox, which says as we make a technology more efficient, we use it more, thereby increasing use of resources. I recently saw a fantastic framing of the Jevons Paradox:
Making use of a resource more profitable increases the return on investment of exploiting that resource.
So increased systems efficiency increases, not decreases, resource use.
Unless your approach is to have a fixed expectation for capability and just optimize for lower and lower resource use to achieve that capability. Which is not how capitalism works, at all.
I agree that this problem is almost impossible to solve under capitalism, capitalism needs to go, but let’s be fair, it’s still a very difficult problem for any alternative economic system. I’d say we should reject any alternative economic system if it doesn’t have some answer for this formulation of the Jevons Paradox; it seems like a good test for whether it can handle existential threats like the climate crisis.
Ahh, the “dissonance” is you’re thinking of the Jevons Paradox as being about increased energy use, when it’s more specifically about the increased usage of goods and services, not necessarily energy.
You’re on the right track, but e.g., if LLM use increased 10x, but we’re able to make them 20x more efficient, the energy use will drop, despite Jevons. Alternatively, Jevons could drive 20x more carbon-neutral solar/wind development making the increased energy use of LLMs (and everything else) a moot point. This is what I think (and hope) will happen, even under capitalism.
It’s unclear to me that the Jevons Paradox would be as much of a problem under other economic systems. To use its original observation about coal: yes, ordinary citizens would continue to want coal and electricity, but much of the usage was by capitalist-driven industry. Companies had to use more coal to ensure they weren’t out-competed by those who did.
A more democratic economic system could weigh the trade-offs, and voters could decide how quickly and how much to adopt a new tech.
Ah, yes, ignoring all of the negative externalities (like pollution), massive capital investment in data centers, LLM companies not turning a profit, etc. we can pretend the slop machines are really cheap!
If a drug dealer offers me the first hit for free, that’s free drugs, that’s fantastic, I love free drugs.
score 2 · canonical
Isn’t it pretty clear though that for something like a security scan or similar that even the externalities included price far cheaper than the alternative? Whether that’s a good thing or not can still be an open question but I’d be interested in hearing the argument that LLMs are only cost effective for finding vulnerabilities because tokens are subsidized/should cost more for some other reason
score 1 · canonical
The Enron-like accounting of LLM companies alone should be enough to make these prices unclear. https://news.bloombergtax.com/financial-accounting/big-tech-ai-spree-revives-accounting-devices-that-toppled-enron
One negative externality is the risk of global financial collapse. So much money is tied up in LLMs and they’re so fishy.
score 1 · canonical
I’m not disputing that the prices as is may not reflect the true cost of the tokens, or anything like that. What I’m trying to get at is that I think it’s clear that LLMs are extremely good at this sort of thing, so even if it were true that the price of the tokens were 100x off, for this particular use case it still might be viable given the counterfactual is something like “spend a lot of money paying pentesters”.
I agree with you that 1M on pentester salaries is likely to produce a better result than 100 dollars to whatever your favorite LLM provider is, but what I’d be interested to know is what you think the breakpoint there is and how far off current token prices are from their true cost
score 2 · canonical
I’m pretty sure spending a million dollars on pentesters produces a whole lot more pollution than $100 on tokens.
score 1 · canonical
When you compare emissions from a computer to emissions from a human, this is a false comparison.
The human already exists and will be producing more or less the same emissions no matter what work you assign to them. However many pentesters you hire, if they are working on pentesting instead of e.g. web development, there’s negligible difference in emissions. Even if they are unemployed and sitting around at home, they will produce similar emissions, perhaps slightly less if they lose their work commute and/or videoconferencing.
The only way you get rid of the human emissions is if you kill the human. The only way to drastically reduce a human’s emissions is to drastically change their society, e.g. by destroying all fossil fuel generators, or by deporting them to an undeveloped country with much lower emissions (changing which society they exist in). Do either poorly and it’s just a fancy way to kill the human.
But you could turn off the LLM data centers and shut down their gas turbines, without killing sentient beings, and their emissions would stop. Humans survived without LLMs only a few years ago, while fossil fuel infrastructure has been baked into our society for over a century.
It feels like you didn’t really read or understand what I said. $100 in tokens is not the true cost. Let me rephrase:
So, salaries for human pentesters cannot be compared to tokens for slop machines.
I also don’t think you can get the same results as $1 million on pentester salaries with $100 in tokens, but whatever.
score 1 · canonical
You will certainly get an equivalent security analysis for orders of magnitude less than $1m in tokens though.
Anyway, your alternative scenarios for humans (pentest or loaf at home) leave out the obvious: move humans away from activity where computers do it better, and have them do something more productive, like high-level architecture and design. It’s the same logic as switching to power drills and cranes, instead of making artisanal woodworking.
Your focus on the costs without mentioning the benefits, assumes LLMs are useless (given you call them “slop machines”). But since they have workable use cases (and in the case of security, they clearly do), then it becomes like any other tech, where we weigh tradeoffs.
I personally think the environmental harms will be short-term, and mitigated over time in a variety of ways. More efficient models, cheap software rewrites into more efficient langs, increased model efficiency from more software, model-driven research into environmental tech, etc.
(Now, whether humanity would actually build the current amount of stuff more efficiently, or just build more is a separate question. But that’s much older than LLMs, and is more your-underlying-problem-is-actually-capitalism again, which cannot be solved by addressing the symptoms instead of the root cause. But that’s a whole 'nother conversation.)
score 1 · canonical
There is some very strange cognitive dissonance in your comment. On the one hand, you say that efficiency will solve the climate impact of LLMs. On the other hand, you recognize that efficiency won’t solve the climate impact, in the same comment! You handwave the problem with efficiency away by blaming it on capitalism, but who cares why efficiency doesn’t work if you’ve just invalidated your own solution?
For those not familiar, we’re discussing the Jevons Paradox, which says as we make a technology more efficient, we use it more, thereby increasing use of resources. I recently saw a fantastic framing of the Jevons Paradox:
I agree that this problem is almost impossible to solve under capitalism, capitalism needs to go, but let’s be fair, it’s still a very difficult problem for any alternative economic system. I’d say we should reject any alternative economic system if it doesn’t have some answer for this formulation of the Jevons Paradox; it seems like a good test for whether it can handle existential threats like the climate crisis.
score 3 · canonical
Ahh, the “dissonance” is you’re thinking of the Jevons Paradox as being about increased energy use, when it’s more specifically about the increased usage of goods and services, not necessarily energy.
You’re on the right track, but e.g., if LLM use increased 10x, but we’re able to make them 20x more efficient, the energy use will drop, despite Jevons. Alternatively, Jevons could drive 20x more carbon-neutral solar/wind development making the increased energy use of LLMs (and everything else) a moot point. This is what I think (and hope) will happen, even under capitalism.
It’s unclear to me that the Jevons Paradox would be as much of a problem under other economic systems. To use its original observation about coal: yes, ordinary citizens would continue to want coal and electricity, but much of the usage was by capitalist-driven industry. Companies had to use more coal to ensure they weren’t out-competed by those who did.
A more democratic economic system could weigh the trade-offs, and voters could decide how quickly and how much to adopt a new tech.
score 1 · canonical