Technology

I dunno why everyone always jumps to accusations like this.

Well, I can't speak to everyone, but I jumped to that conclusion because you proposed the idea that the current top of the line models are simply big fat transformers and nothing else.

I'm really glad to hear that you've had your fingers in the pie, and have a good grasp of what an LLM is and what it does on a mechanical level. It makes talking about them much easier. I'm super cool with continuing a discussion if you are interested in why I think they are an integral part of eventual AGI.

To clarify, I do not think that their current form is a 1:1. I disagree with your blimp analogy though. I think a better comparison would be an internal combustion engine. It's certainly not a turbofan engine or a scramjet, but the basic concept is there.

Turning language into a fuzzy world model that can be interfaced with simply, and performing math on that model, is just as important to AGI as a fully fleshed out physics simulation is for AGI. Humans have both. Why would an AGI not need them?

What an LLM does to it's array is not "intelligence". It's simple math. However, it is performed on a thing so complex that it cannot be derived from any math we have now, a manifold created through eons of interface between all humans and the universe.

The "magic" isn't in the simple math, it's in the thing we all made together for millions of years. An interface with this, even one as "simple" as an autoregressive transformer, is an indespensible part of AGI, both because it provides a fuzzy way to understand the universe, and because it provides an interface with humanity, and humanity processes the universe in a way that is so complex, computers will not be able to emulate it for generations.

Edit: also, fuck altman and musk and zuck and even amodei. I wouldn't trust anything they say either.