This is a most excellent place for technology news and articles.
The main issue with Open / Local LLMs is the lack of broad support. It's not hard for me to run Local LLMs, but apparently a decade of informal python experience and using linux mint is intimidating to some people. Apparently people want to click button and talk to phone god? Mostly speaks of a failing education system if you ask me.
I'm not even that good with AI, I just lurked a bit on !LocalLLaMA@sh.itjust.works and got mine running at like 30 tokens/second on my laptop.
Thats why I'm actually at least for now hoping for the FOSS Chinese Frontier labs to make the breakthroughs since many of them still share research and open weight models
Most of us here think streaming is useful, despite the fact that streaming services are based on the idea that they own and control all access to media with you renting it from them, raising prices, shoving ads in your face, and removing media from the collection you’re renting from them. Our solution in this community is investing in the open source community around Jellyfin, the Arr stack, etc. so we can still enjoy the benefits of streaming services while owning our own content and not having to financially support companies we don’t agree with. I’d say a lot of us here who are happily using Jellyfin might otherwise have a streaming account if it didn’t exist. Admittedly, I used Netflix until I realized I had better options.
I think the same is true with local LLMs. Not all of us agree that LLMs are useful, but most of us here agree that a few tech giants tightly controlling LLMs and renting them to everyone is not going to be a good thing. Without self-hosted LLMs, many people who do find value using them will go ahead and financially support the rent-seekers who are hell bent on destroying the world for their own financial gain, as well as support them by sharing data that can be used to train their models. Even when you use the free tier of Chat GPT, for example, you’re supporting OpenAI by giving them your prompts that they can use to make their models better.
The ecosystem around running open weight models is rapidly evolving. I’m already running the Qwen 3.6 MoE model with the desktop beta of OpenCode on my gaming laptop and it’s pretty decent. Personally, I’ve found ways to use LLMs where they are actually useful and not just slop generators, though I initially thought they were useless before I spent a lot of time working with them. I’m all for supporting and contributing to this ecosystem so that people can use LLMs without giving their money and data to shithead psychopaths.
👏 This is the way to do it!
