219
AI godfather Geoffrey Hinton: ‘We’re creating alien beings. I’m willing to be selfish and choose humans’
(www.theguardian.com)
this post was submitted on 13 Oct 2026
219 points (94.3% liked)
Technology
88783 readers
2650 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
There's some semantics here.
Large neural networks have been shown to build parallel models of the world that they learn through their training data. A chess model trained only on chess move notation does have an interior board and state that it tracks, despite never being told about a board, the number of pieces, how they move, etc.
LLMs do something similar with language, mapping tokens to ideas in ways that can produce meaningful and grammatically correct output without needing to be formally taught any of this.
Our brains also do things like this - we observe X and then try to intuit Y, the truth of the world and reason for X.
Where this breaks down is humans can then take our guess for Y and propose Z, and try to find it out in the world. If we can't, then Y is probably wrong, and our new understanding of the world is Y* that is consistent both with X and the absence of Z.
I cannot stress enough how far we are from machines that can do this. They can learn to approximate Y through training, but they have no means to extend that to propose an unobserved Z, no means to update their internal state given the results of that observation (without fully retraining the model on new data - which makes it a new model), and no way to question X given Z - they can talk at great length about data biases but can literally never excise bad data from their training weights. There have been experiments with training models on time-boxed datasets (e.g. stuff up to AD 1880) and asking it about modern concepts - predictably hilarious results.
All the current 'AI' stuff is smoke and mirrors around this very hard problem. The machines can talk, but they can't think (Z) and can't learn (Y*).
And you can't excise 20 years of Fox News from some boomer conservative. They can literally never back down from their biases.
Training data is a lot more permanent than context, but at the same time, it's not absolute, either. I've seen plenty of thinking blocks where it knows about X from its training data, but perhaps it's out-of-date and should check other sources. It has the web tools, finds newer information, processes it as part of its problem set, and solves the problem with newer information.
The core issue is we throw all of this data away when we're done with the problem! A LLM only has so much context, so it starts to compact after a while. And even if it doesn't, we voluntarily chuck that shit out the window, as soon as the session ends.
It's like Lenny from Memento. You got a baseline person who constantly believes he's got to find his wife's killer. And even when he tries to supplant his lack of long-term memory with photos and tattoos, it's a shit system with shittier outcomes. Also, he doesn't get to choose when his memory disappears. It just happens with some kind of stress trigger. Because of how flawed Lemmy is, do we call him somebody who can't think and can't learn? Is he intelligent? Is he even sentient?
Though, in the case of an LLM, it doesn't have to be set up this way. We can still stash a long-term version of memories as rules and other permanent files. In fact, this is encouraged. Everything starts with a system prompt, a set of rules, and the user prompt. All of that is adjustable.
And yes, you can train out biases. Rather easily, actually. You don't even have to adjust weights. You post-train through rules files.
But, in terms of philosophical questions, can we call LLMs intelligent if we can edit their memories at a whim? Is a brainwashed boomer intelligent because Fox News hacked their memories and biases? Is Lenny intelligent because he can't form new memories?
You have to remember that just about every argument for or against LLMs can also apply to humans, and you can't really arrive at definitions without clearly separating what doesn't apply to LLMs and does apply to humans, in all cases.
The assertion is not about whether 'you' can do it. It is that the model cannot do this itself.
A human recognizing a bias in the model and correcting it is not evidence robot intelligence, it is further evidence for human intelligence.
Training is absolute, it determines the set of relationships the model can represent between input and output. Context, rules files, system prompts, etc. operate only on the conditional probabilities within this set and are impermanent. This is 'priming' in human psychology, and why a person in a herpetology class will talk about Pythonic features differently than in a coding class.
As an example of how this is not real learning and is inadequate to solve real problems, here is a paper where the authors challenged multiple models to solve unfamiliar games where they need to discover the rules through play and feedback. Even the 'best' model performed pitifully compared to humans.