This is a most excellent place for technology news and articles.
Whoever thought that this machine that can predict next word in a sentence, next sentence in a conversation etc. should be used in place of all human intellectual work... should have his elderly care taken over by LLM.
That's what is wild about it. At any given point in time, the model is wholly consumed only with the very next token. Maybe that token is a running narrative of 'reasoning' or directly in the output, either way, the AI does not have anything to model anything beyond the very next token. It doesn't have a destination in mind and is just finding the words to get there, it's building it up word by word. The overall 'meaning' is an emergent property of just picking the very next token and seeing what happens.
Honestly, it's shocking it works as well as it does. More shockingly, there are AI enthusiasts that argue that's how the human brain works, which I can't imagine someone going through life with every thought rooted in building it up word by word.
Here here! well said!
Many dystopian science fiction books have already become reality. Look at "Enemy of the State" for example.
Heck, back in like what 1971? Enders game predicted the whole internet Influencer concept.
fwiw Ender's Game was published in 1985. (Based on a short story from 1977, but that story didn't have that subplot with Peter and Valentine.)
Voice to text strikes again!
Ah, got it! I can remove the link or leave it up for others' reference. Let me know if you have a preference.
In 1897, they built the first music synthesizer. It worked, but it took up the basement of an entire city-block sized building, so it was essentially useless. After a few decades of development, it could fit in a suitcase, and be carried around.
Data Centers are like that 1897 synthesizer. Sure, it works, but at what cost? It clearly isn't ready for prime time. Go back to the drawing board, tweak the problems, including regulations, and maybe in a couple of decades, we take another run at the new and improved version.
It's not even that it isn't 'ready for prime time', largely to the extent it works, it works with not so crazy requirements.
The problem is that they don't settle for what it is, they try to overextend it. In software development for example, in the cases where it works at all, you are 95% of the possible successes within 3 or 4 iterations. Problem is they demand that extra 5% which takes an order of magnitude more. Note that '100%' here is the max success possible with AI, not 100% success in general, that number varies greatly with context. So it might be in a certain scenario more like going from 19% AI curated to 20% AI curated, at huge incremental expense.
Data Centers are like that 1897 synthesizer. Sure, it works, but at what cost? It clearly isn’t ready for prime time. Go back to the drawing board, tweak the problems, including regulations, and maybe in a couple of decades, we take another run at the new and improved version.
The issue with AI is not a technical or development problem. It's not even a regulation problem. It's a capitalism problem. Infinite growth will still be as unscalable in a hundred years as it is now no matter how good and mature the tech is.
There are also hard mathematical limits stalling AI growth. Frontier models haven't improved in like a year despite being fed money by basically the entire global economy. Diminishing returns on steroids basically. They're already at the limit of what they can make, and going further gives a much smaller improvement in the model, and now I hear there might not be enough human written material on the internet to train them.
It also looks like hallucinations are inherent to LLMs and you can't get rid of them. It's a side effect of the model. What commercial applications are there then, if you can't guarantee the output? It's worse than a human for most things since it doesn't know truth from lie and will confidently say both as if they're fact. It also looks like prompt injection isn't something you can fully guard against either.
What's the value proposition when you can't trust the output and the model might give a massive refund or discount to a customer and the courts rule the AI speaks on behalf of your company?
I feel like calling „hallucinations“ a side effect isn’t really describing the issue properly. They’re not a side effect, nor are they hallucinations. It implied that there’s somehow something that distinguishes „correct“ output from „incorrect“. There isn’t, it’s all just output. The output resembling actual factual reality is statistical chance.
It’s worse than a human for most things since it doesn’t know truth from lie and will confidently say both as if they’re fact
It works for most executives and sales folks.
Baseless confidence is the recipe for business success, which is why they love these AI chatbots.
Bigger problem for the business leaders is how sycophantic they want to be to the user. If an insurance company used it for claims, it might actually approve a claim, and that would be unforgivable for them.
Diminishing returns on steroids? No, clearly we just need to pump EVEN MORE MONEY AND DATA into this
If we just vaporize the future of everyone under 60 we can make our auto correct engine 3% less likely to lie out of its ass 🤡
But must make number go up by any means
Chanting:
Number go up!
Number go up!
#️⃣💨⬆️❗
In 1897, they built the first music synthesizer. It worked, but it took up the basement of an entire city-block sized building, so it was essentially useless. After a few decades of development, it could fit in a suitcase, and be carried around.
As per the 1890s, it probably also had to be lubricated with orphaned child blood.
Don't give the AI MAGAs any ideas.
Efficiency deserves more attention than hype.