I usually don’t even understand the lingo they use. “Open-weighted” is the most recent one, then it usually goes down to specific “models” that everybody is supposed to know about.
These are my thoughts (I will stick to the vague “it” for now, but of course therein lies another question: “and how does all this apply to various specialised AIs”):
- Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?
- If yes to the previous: the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?
If what I wrote above is true, what exactly are people arguing when they say it’s still possible to use LLMs ethically or true to FOSS philosophy, because … ???
edit
Thanks to all who answered.
I guess it’s my fault for asking several questions in one, but this thread has attracted exactly the type of people I’m writing about; several even used the term “open-weighted models” without explaining it.
Asking to get arguments explained, I got more arguments instead.
You can run them locally, yes. There are models that can even run on phones, but usecase is limited. But it can only be considered ethical, if the training data used is listed or ethically sourced IMO.
AI bros on Lemmy will disagree with me, but most open weight models are still trained unethically i.e, theft. Most proponents of LLMs (who I talked to on bsky), who say local models are ethical, don’t fucking use it. They’re larping on socials about how awesome it is, but none of the ones I talked to are using it in their projects. They mess around, realise it is not as good as the “unethical” options, go right back to Claude
Open weight models Qwen, deepseek, mistral, and the Ollama stuff etc are unethical in normal people’s eyes, but “ethical” enough for AI bros.
From what I searched, there are very few that can be considered ethical - Olmo, Apertus, Starcoder(?). But idk anyone who uses these. My friend at IBM said they used Apertus, but it was nowhere near good as ChatGPT, so they no longer use Apertus now. And these models require minimum 6-8 GB VRAM for their lowest parameter model iirc.
Even the open-weight model bros are lobbying to redefine what ‘open-source AI’ means. That should give you a fair idea about people behind open-weight as well
Borderline strawman there but I’ll bite.
Open weight models trained unethically are unethical. Closed weight models trained unethically and then sold back to you for a profit from gas-powered datacenters funded through Ponzi schemes are substantially more unethical.
From there it’s a harm reduction calculus. No, those are never pleasant.
So do you let the closed weight labs conquer the field unopposed just so you can feel better about yourself? That’s a valid stance, FWIW, and it’s also super easy and convenient because you don’t have to do anything. It especially makes sense if you believe it’s still possible that AI will just go away on its own. I don’t, myself, not anymore, so I encourage the use of open weight models, however grudging, so people don’t give money to the closed labs and in the worst case aren’t eventually stuck with the maximally unethical options. And we’ve not even touched on the nightmare labor replacement scenarios that seem every day less unlikely. Am I right? I have no clue. Like you, I’m just trying to make the best choices I can in a world that’s gone to shit. I’d recommend dropping holier-than-thou attitude either way, though, because it doesn’t help our side. Man.
Just as I suspected…
Thanks for taking the time.
This is software meant to be run always and completely locally?
Sorry to whine, but so far nobody has eli5’d what “open-weighted” means, or “model” at that… please?
That’s what is claimed, I haven’t run them locally since I don’t have a good system.
To be honest, I’m not sure if I can eli5 weights and models, but I’ll try. Think of a model like the base - for example, OpenAI has different models like Astra, Sol, etc. These are different models, like different versions of a software or operating system like macOS, but for AI stuff. Like one would download a software, you download a model to perform tasks.
Weights are vales that can influence inputs of these models to get a desired/better result. What most of these models are doing is mostly predicting what might be the next appropriate text/data to the question you asked. When you ask these AI models what 2+2 is, it is not performing a math operation like a normal program, it is looking at its training data to see what the closest option might be. It is doing pattern matching.
These AI models inside can be thought of like an interconnected network, like neurons in our body, that keep passing information to the next neuron and to the brain to make a decision. (Before understanding LLMs it would help to understand Neural Networks first). These AI networks need weights and biases. These networks perform calculations and weights are used to determine how much importance/weight each input can have on the output. Bias on the other hand, is used to shift/change the output so the AI model can ‘learn’ to pattern match better.
What open-weight models, do is they make the model available for download along with the weights. No information is given on training data. Like with ads, ones with most data emerges victorious i.e, has a better model. So these companies do theft, don’t list their training data afraid of getting caught. I forgot which one, but either Deepseek or Qwen (both open-weight) was caught ‘stealing’ from Claude (not open weight). You can probably guess how much these companies value ethics.
I’m not sure if this entire thing goes away, but local models might be the ones left standing when this bubble pops.
Open weight is different to open source. Open Source AI as it stands, the definition requires a model to have entire thing made public - so the weights, biases, training data used, the model. Apertus, Olmo etc are mostly meeting open source AI definition.
If you need to know more, this is what we’d use to refresh our memory before exams :)
I probably might have made mistakes here, English isn’t my first language either. But I hope you get an idea about these terms
If you really need to understand this tech more, I recommend watching ‘AI for Everyone’ course on Coursera from Andrew Ng. It is free to audit, my friends who took his course were hyped (I wasn’t really interested in AI)
I would unironically argue that a model primarily trained through distillation of closed frontier models, and then released open-weight with an open-source architecture, becomes “ethical” again.
Something something Robin Hood
Rob the poor’s money from the rich and keep it for your community?
Sounds more like feudal warfare than anything, I can’t see any harm to artists being reduced at all