While Taiwan’s situation is obviously a bit extreme, I’m pretty sure there’s a lot to learn for everybody.
Internet blackouts and censorship are sadly becoming more and more common around the world, and as one blogger in Iran put it while experiencing one:
Many of our software are built on top of the assumption of good, stable, high speed connectivity, up to date hardware and the ability to babysit, update and fix the software constantly. In bad situations like this, you don’t have any of this. You might go months without being connected to the internet, you might have to use decades old hardware, and the software should be reliable enough so you don’t have to constantly babysit it.
An internet blackout where your own government will crack down on connecting to the outside world is also a rather extreme situation compared to not being able to load a bloated react app on a 2G connection or from Antarctica, but the point remains that much of the software we rely on is quite bandwidth and compute hungry.
But most orgs, outside of governmental intervention, are not incentivized to care about things like this. I feel there are some individual efforts that could be very helpful (having one’s own simplified search engine like ~technomancy does instead of just solely relying on google for example), but if everyone’s insistent on running critical comms through WhatsApp or Line or Facebook Messenger which go down with the submarine cables / us-east-1 outage, there’s not really much you can do.
Well, besides knocking sites offline AI scrapers have also forced half the web behind captchas or anubis challenges. Which are terrible on low bandwidth, and make it basically impossible to use browsers such as lynx.
Why do you think it would be worse? I’m seeing more local software, more things freed from SaaS jail, people reverse engineering proprietary drivers, etc. On top of that the barrier to entry for emergency networks like Meshtastic and overlays like Iroh is lower than ever.
We have local models capable of a lot right now and it’s only getting better every day. The hardware requirements are not great, but not absurd either - you can still share community resources, like when you need a lathe, there are people around with garage workshops.
First of all, the baseline is not great, so if things needed for users are whatever-local-LLM-ed by people with a general idea of what is going on instead of previously being blocked on lack of energy to do it properly, and the quality is above the pre-LLM corposlop (not always a high bar), it might still end up an overall improvement.
But the point you make is important regardless of this balance.
I would say that in terms of usefulness without fixing the code by hand the last few months have been a large change for local-ish models, and maybe in a few months there finally is a model that is reasonably local (good speed on 8 GiB RAM + 16 GiB system RAM + NVMe, dunno — I run on iGPU which shifts balance weirdly), and also not useless for generating single-usecase applications without looking at code at all.
LLM+diffusion image generation seems to be run by power users (sure, you need to install something then go download a plugin and install it correctly than go download a model and put its pieces into correct directories, the learning curve is there) who do not know any programming quite a lot.
Probably once it gets the same in terms of result-tier/performance/requirements for mini-app-gen as for image-gen, in a few months it starts approaching the same situation in terms of usability, harnesses included. Without harnesses Ollama is already pretty friendly, I think.
Internet blackouts and censorship are sadly becoming more and more common around the world, and as one blogger in Iran put it while experiencing one:
An internet blackout where your own government will crack down on connecting to the outside world is also a rather extreme situation compared to not being able to load a bloated react app on a 2G connection or from Antarctica, but the point remains that much of the software we rely on is quite bandwidth and compute hungry.
But most orgs, outside of governmental intervention, are not incentivized to care about things like this. I feel there are some individual efforts that could be very helpful (having one’s own simplified search engine like ~technomancy does instead of just solely relying on google for example), but if everyone’s insistent on running critical comms through WhatsApp or Line or Facebook Messenger which go down with the submarine cables / us-east-1 outage, there’s not really much you can do.
score 1 · canonical
Imagine how bad it’ll be with the incoming torrent of slopware!
score 1 · canonical
Well, besides knocking sites offline AI scrapers have also forced half the web behind captchas or anubis challenges. Which are terrible on low bandwidth, and make it basically impossible to use browsers such as lynx.
score 1 · canonical
Why do you think it would be worse? I’m seeing more local software, more things freed from SaaS jail, people reverse engineering proprietary drivers, etc. On top of that the barrier to entry for emergency networks like Meshtastic and overlays like Iroh is lower than ever.
score 1 · canonical
… all dependent on a SaaS for construction and, more importantly, likely all future maintenance. Fun!
score 1 · canonical
We have local models capable of a lot right now and it’s only getting better every day. The hardware requirements are not great, but not absurd either - you can still share community resources, like when you need a lathe, there are people around with garage workshops.
score 1 · canonical
Those local models are essentially only used by people who know how to program in general, so this adds very little.
score 1 · canonical
First of all, the baseline is not great, so if things needed for users are whatever-local-LLM-ed by people with a general idea of what is going on instead of previously being blocked on lack of energy to do it properly, and the quality is above the pre-LLM corposlop (not always a high bar), it might still end up an overall improvement.
But the point you make is important regardless of this balance.
I would say that in terms of usefulness without fixing the code by hand the last few months have been a large change for local-ish models, and maybe in a few months there finally is a model that is reasonably local (good speed on 8 GiB RAM + 16 GiB system RAM + NVMe, dunno — I run on iGPU which shifts balance weirdly), and also not useless for generating single-usecase applications without looking at code at all.
LLM+diffusion image generation seems to be run by power users (sure, you need to install something then go download a plugin and install it correctly than go download a model and put its pieces into correct directories, the learning curve is there) who do not know any programming quite a lot.
Probably once it gets the same in terms of result-tier/performance/requirements for mini-app-gen as for image-gen, in a few months it starts approaching the same situation in terms of usability, harnesses included. Without harnesses Ollama is already pretty friendly, I think.
score 1 · canonical