735
Generative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.
(www.theatlantic.com)
this post was submitted on 15 Jul 2026
735 points (97.4% liked)
Technology
86401 readers
3475 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Whoever thought that this machine that can predict next word in a sentence, next sentence in a conversation etc. should be used in place of all human intellectual work... should have his elderly care taken over by LLM.
That's what is wild about it. At any given point in time, the model is wholly consumed only with the very next token. Maybe that token is a running narrative of 'reasoning' or directly in the output, either way, the AI does not have anything to model anything beyond the very next token. It doesn't have a destination in mind and is just finding the words to get there, it's building it up word by word. The overall 'meaning' is an emergent property of just picking the very next token and seeing what happens.
Honestly, it's shocking it works as well as it does. More shockingly, there are AI enthusiasts that argue that's how the human brain works, which I can't imagine someone going through life with every thought rooted in building it up word by word.
Well yes and no. it is steered by a buffer of context as well that sorts/ranks/informs what the next word should be. That context differentiates if you are talking about apple the fruit or apple the company or apple the device. Heres a great overview if anyone is interested. And no, its not my video. Its a youtube intro to how AI works. Best watched with duckduckgo browser which trims out youtubes overly frequent ad interruptions. https://www.youtube.com/watch?v=OYvlznJ4IZQ
But that context is a mix of model output and other sources. The model output portion was generated token by token, and is combined in interesting ways with things like human response, search results, software output. It's still a backward looking mechanism, rather than having established a concept as a goal and then trying to build the words to reach that concept like we do.
Size and strategy for managing the context has been critical for improved subjective results, but it still doesn't exhibit the behavior of the words as a tool to address some concept, everything about the model is about the words themselves. So we end up with something very good at generating what seems right and there's a super high chance of what seems to be right actually being right. Especially when the software can automatically execute commands and the good or bad results reach into the context window, enabling it to effectively get automatically second guessed. The potential for automatic verification in some scenarios automatically feeding the context window is what makes it particularly appealing for software folks, though not universal.