Ask Lemmy
A Fediverse community for open-ended, thought provoking questions
Rules: (interactive)
1) Be nice and; have fun
Doxxing, trolling, sealioning, racism, toxicity and dog-whistling are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them
2) All posts must end with a '?'
This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?
3) No spam
Please do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.
4) NSFW is okay, within reason
Just remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com.
NSFW comments should be restricted to posts tagged [NSFW].
5) This is not a support community.
It is not a place for 'how do I?', type questions.
If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.
6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !uspolitics@lemmy.world
7) No Hit-and-Run questions.
Please don't delete your post for no apparent reason. If you plan on deleting a question later, say so in the post, or if you feel that you have a good reason to remove it, message a mod beforehand. It's not fair to the ones who took their time to answer, and it's not in the spirit of the community.
Reminder: The terms of service apply here too.
Partnered Communities:
Logo design credit goes to: tubbadu
view the rest of the comments
Okay, yes, we aren’t doing single words at a time (technically some models do chunk long words anyway so even from near the beginning we weren’t doing “singular” words at any time)
However, you are both right and wrong in your assertion that LLMs our equivalent human responses.
The translation of thoughts to words and words to motor functions both can be approximated by ANNs. And yes, the work done to select words we use is a probabilistic process like you describe. We hear patterns in language and that makes us more likely to use that phrasing. The more you hear a phrase the more likely you are to use it over another and when two or more phrases would communicate what you want to say your brain basically just picks one.
So, If speech production (or sentence construction) was what you meant by saying “it’s the same for human responses,” then yes, we agree. Both are probabilistic word generators and likely work in similar ways. (In fact I think place cells were found in Wernicks area (?) or one of the other speech corteces which means some of our word selection is likely similar to the results from transformer architectures)
However, if you meant the entirety of human response—as in from hearing/reading a comment, thinking about it, responding—is the same as current LLMs generating text. I strongly disagree.
The actual process of “thinking” is not something an ANN (especially a non-recurrent one) can do. The ability to ruminate on thoughts and make changes/learn-new-things simply by trying to formulate ideas before even deciding to comment cannot be accomplished with a pre-trained static net, not even one with memory or the illusion of memory like current LLMs. (Not to mention that identity also plays a large role in our responses and it too cannot arise from current deterministic architectures)
As for me asserting human response is chemistry is more like asserting AI is electromagnetism, there are many reasons why, but the simplest illustration would be this:
This is what I meant.
That would be crazy, I'm glad you would disagree with that.
This is moving into a funny gray area, but what you are talking about is, I think, only possible if you take a route like the one covered in Jeff Hawkins' "A thousand brains". It's not the most fun read if you're not into neuroscience, but the second half is pretty relevant regardless.
Deleted: accidentally posted same comment twice.
I haven’t read the book but I am familiar with the thousand brains hypothesis. The real problem as far as I can tell seems to be the variations in morphology and connectivity of different neurons.
The brain might make every column the same to begin with but if that’s the case, the diversity of the initial columns is immense. So many different genes even for just pyramidal neurons. Not to mention the inter neurons and other glia.
Plus the function of many cells are still unknown like chandelier cells. They’re everywhere, they regulate firing, but we’re not sure how. They can be inhibitory or excitatory and sometimes they can fire in response to both inhibitory or excitatory input, Etc.
And don’t even get me started on how no one actually seems to agree on the function of the layers of the neocortex. Every paper I read on the topic poses almost entirely different hypotheses for the function of each layer and the few connection maps you can find will show many connections that violate the idea each layer takes specific inputs.
Sure spiking networks are much more biologically plausible (and fun to work with so I recommend you try one out if you’re interested in this field) but the connections and learning rules are what seems to matter more.