Mildly Infuriating

AI is already expected and required in my job

then you should be looking for a new job... unless you happen to be on a very specific niche, any company so committed to current AI that cannot survive without, will implode in short time

Speaking as someone with what I’d consider to be a pretty decent homelab, I can’t self host the kind of AI that I use for work, not without literally emptying my savings into the hardware and energy needs.

I think we’re at least a decade away from having user-accessible hardware for AI that doesn’t break the bank.

So, for coding, which is what Gates was specifically talking about being doable now, maybe we could corral up the hardware. Like, maybe one could cover specific fields.

There are still going to be issues like power and cooling and hardware cost and whether people who make competitive models are even interested in providing it for home use (since it makes it harder for them to make a return on their model). But set that aside.

As I've said before on here, I would say pretty confidently that we will not have local models running to do all of the stuff that cloud compute is used for or is being built out to for at least something like four to five years, and that's if we started immediate, massive buildout of memory fabrication to a much greater degree than we have. You cannot build a new memory factory in less than that timeframe, and we will not have that capacity with existing factories. The majority of fabricated memory now is going to cloud AI use, and cloud AI hardware will have considerably higher capacity utilization than hardware at home. You'd have to have many times over as much memory being produced to have the same compute capacity at home.

I'm not opposed to doing LLMs or parallel compute at home at all. I have a 128GB Framework Desktop and an XT 7900 XTX that I got to do just that. I'm just saying that we are not going to realistically be able to move all of the stuff in the cloud to the home for at least something like half a decade, and very probably more, because humanity does not have the memory available and can't build enough memory fabrication capacity for it in that timeframe. It doesn't matter how much value is being provided by some home user of that hardware or what their willingness is to spend on it if we don't have the memory. Like, even if every person in the world could produce, to pull a number out of the air, a real $1M in value every year via use of a home AI rig, even if all that demand suddenly materialized out of thin air, all that would happen is that prices would rise sufficiently to make the hardware unaffordable even at those extreme levels. The constraint is on the supply end, not the demand end.

Understandably you won't be running meta level Machine models, but running olama and downloading a open LLM model can get users started for simple things they might have already been doing.

I am running a fully local model for my smart home needs, since Google's killed of Assistant and is now pushing Gemini down my throat, I figured I would just do it my self.

For anyone interested take a look at NetworkChuck on YouTube. His videos are a little to sensational for me, but he does have some interesting topics covered from time to time.

https://www.youtube.com/watch?v=QQEgIo4Juxg