207
Linus Torvalds used AI to fix a Linux bug, and now LLM critics on Linux face an uncomfortable choice
(www.xda-developers.com)
This is a most excellent place for technology news and articles.
If the model is adequately FOSS, i.e. open weights, and can run on a single consumer GPU (or NPU), and the "author" (quotations because I'm personally undecided if one can claim code generated by an aforementioned model is theirs) understands it, then I really don't see what the problem is...
Personally I have nothing against SLMs/LLMs as a technology, to me my grievances against ChatGPT or Claude are mostly about their environmental impacts and selling us back our own art, also keeping knowledge behind a for-profit black-box - if those aren't appropriate for a specific model, then I say using that model is fair, and good for productivity.
I'd take a model whose training data is open source and legitimately obtained. The only ones I know about are Apertus and OLMo, and they aren't really competitive.
Unfortunately not yet, no, true FOSS models are likely many years away, but I would argue that that follows typical FOSS lifecycles. Emerging technology is typically outperformed by proprietary endeavors, which creates an audience, and then that audience undertakes a FOSS implementation that initially underperforms, then just about competes, then eventually overtakes (i.e, GNU/Linux).
The problem with open weight models is that they are still hard to deploy. For DeepSeek for example you require NVIDIA HGX B200, these things are expensive.
If you are really using those things for work, a 32GB VRAM Blackwell currently costs around 3800-4000€. My own experience with local models on my 5070/12GB with 64GB DDR4 has shown me:
If this scales linear, it would mean that the 32 GB of a smallish Blackwell would be enough to run Ternary models with around 70B parameters plus a ton of context window without breaking a sweat.
These are strange times. I absolutely love the technology and the advances, and i fucking hate the companies pusting these technologies.
Altman, Amodei, Musk, Nadella, Huang and all of the othe AI evangelists can go choke on a bag of dicks in my opinion. This technology never should have been this divisive, it should have just become another tool in humanities toolbox. I fear that these people have tainted the general populations outlook on this tech for a very long time.
Yes, unfortunately they are, and I do think efficiency is going to be a significant research front for open-weight models. The nature of this topic is highly speculative as our compute capabilities have only recently reached what is required to consider running generative AI models, what we have today are very crude first implementations of what I personally believe will become an everyday tool for developers, and more.
And we have seen this, there are models now capable of running on an individual's hardware (and not particularly expensive hardware either) that can outperform what ChatGPT initially launched with.