This is a most excellent place for technology news and articles.
But it’s so good at programming if you already know how to program! Surely that’s worth burning the planet and crashing the world economy??
Actually still no
https://github.com/JustVugg/colibri
Everyone was desperate to be first because capitalism. But we are getting good models without the insane build out requirement. Which will be hilarious to leave the cunts holding the bag. Not that the planet is better for it in the end.
The engine is a single C file (c/glm.c, ~2,400 lines)
That file is almost 6k lines. The style also makes my eyes bleed. Why do people pretend stuffing 6k lines of code with almost no whitespace and meaningless variable names into a single file is a good thing? I've seen this a lot recently
~1 token per second (storage bound gen4 nvme)... Some of us have places to be.
Don't get me wrong. Its impressive that it can run at all, but honestly the usecase is exceedingly narrow. You'd have better results with a structured quantized gpu-only gemma or qwen workflow. Quality over quantity, rely on validation and a structured process: lots of cross-model review and iteration loops with spec and test driven dev. You could probably get a working alpha by the time colibri set up the environment.
Yeah I'm just beginning my local AI journey on a 5080, tried Qwen3.6 27b Q4 and was getting like 1tps because of the vram overflow. Ran it over night at it was still chewing on generating a prompt for a sub agent when I got up in the middle of the night until it simply ended in some kind of "fetch failure" lol. I think I gave it something too large to tackle, but either way 1tps is kinda garbage.
You could use the 35B MoE model, tune it a little bit and get much better results. I have a 5060 ti and 70-80 tok/s are the norm
Wow I’m starting to feel bad about that time I asked AI to make a joke about scatology & eschatology sounding similar.
Yeah this article is already outdated and poorly researched
All for software that'll be out of date and fashion next year!
We are repeating an old pattern in computing: throw more hardware at the problem until efficiency becomes impossible to ignore. Bigger models have delivered remarkable gains, but they’re increasingly expensive. The next breakthroughs may come less from adding parameters and more from smarter architectures, better algorithms and more efficient inference.
DeepSeek has really led the way here, especially as they are a bit more hardware constrained. Plus they openly publish their findings and release open source models, so high hopes there.
It's probably China's play to pop the AI bubble, but I'm all for it (:
I wonder what all is in the deepseek code that is malicious. I'd like to try it but don't want a million Mb/s of tracker shit across my network and can't run it myself.
It isn't about content generation at all. It's about pattern recognition and prediction, which, in the hands of those with the most power to change the world, offers insights into our collective behavior that rulers from every age would have committed genocide to get. AI will tell them how to better build the prison the poor are being impoverished into.
It has the same flaw as every other overreaching evil. We outnumber them. A significant number on our side is willing to kill the other side.
Ima die in the crossfire for sure. But the evildoers always assume they are going to win and they literally never do. They always lose. Expensively.
It’s a dying echo. Nothing more.
Stop spreading criti-hype! Zuck didn’t invent a mind-control ray with targeted advertising, and Sam Altman doesn’t run a terminator factory with GenAI.
if only AI companies optimized their AI to run on less compute (in the data centers)
I feel like literally everyone knows this now but theres so much invested in it theres no backing out.
Too many billionaires have a need to invest and a need for future gains. It's a mental compulsion.
The author seems to be confusing user scalability with performance scaling:
The problem with generative AI, in the industry’s own jargon, is that it does not scale. The cost of growing from, say, a thousand users to a million is a key factor that venture capitalists examine when they evaluate start-ups.
This is a question of whether openai can handle 1 million users asking chatgpt to write a basic html website. That can be scaled horizontally and is just a matter of building more data centers.
The author then goes on to conflate this user scaling with performance scaling:
Yet the returns are diminishing. The bigger an AI model is, the less it improves with each added parameter, and so it must be made bigger at a faster rate just to sustain steady progress. I asked a few AI researchers whether they could name any other real-world software that scales so poorly. None of them could think of any. Even outside the world of software, it’s hard to find a comparable example, given that economy of scale is the principle that has made light bulbs, cars, and clothing so affordable. By economic and engineering measures, generative AI might be the worst technology ever deployed.
This is a question of whether chatgpt can generate a full complex web app. For this there may be a limit to this bigger model approach but this is common to most technologies, performance sometimes has hard limits. You aren't going to get a car to go 300 mph by making the engine bigger and adding more cylinders, there's diminishing returns, that doesn't make cars the worst technology ever deployed... maybe they are but for other reasons.
Economies of scale also isn't about performance scaling, it's about capacity scaling. Capacity scaling for AI does reflect economies of scale, that's why you have these large AI companies building large data centers.
That can be scaled horizontally and is just a matter of building more data centers.
At one of my old jobs "just" was considered a bad word
One does not simply walk into Mordor.
