Technology

It's a quick way to say that the emperor still has no clothes.

We're boiling the oceans to train the models, and these are well-publicised failure modes. If they haven't fixed it, it seems to suggest they CAN'T fix it with the tools and architecture they have. So what other problems is it whiffing on that aren't trivially checkable?

If marketing boxed in the product and said "it does these ten things well", we might be willing to forgive limitations when we leave its wheelhouse. Nobody kvetches that Microsoft Word is an awful IDE, after all. But that would require a retreat from a public that's been promised Lt. Cmdr. Data in your pocket, and investors that have priced it as such.

You shouldn't confront a bullshitter when you catch it in the act!

Al seems to know everything, until it's a topic where you have firsthand knowledge

It's a chatbot that can see and draw, but rich idiots keep pushing it as an oracle. As if "a chatbot that can see and draw" isn't impressive enough.

Three years ago, 'label a tandem bicycle' would've produced a tricycle covered in squiggles. Four years ago it was impossible. I don't mean 'really really hard.' I mean we had no fucking idea how to make that program. People have been trying since code came on punchcards.

LLMs can almost-sorta-kinda do it, despite being completely the wrong approach. It's shocking that 'guess the next word' works this well. I'm confused by the lack of experimentation in, just... asking a different question. Diffusion's doing miracles with 'estimate the noise.' Video generators can do photorealism faster and cheaper than an actual camera.

The problem is, rich idiots claim this makes it an actual camera. In that context, it's fair to point out when a video shows the Eiffel Tower in Berlin. It's deeply impressive that computers can do that, now. But we can't let it guide people's vacation plans.

Because they are promoted as being able to do anything when jammed into everything.

why do everybody want to make it count letters and stuff like this?

Dunno about the others; I do it because it shows well that those models are unable to understand and follow simple procedures, such as the ones necessary to: count letters, multiply numbers (including large ones - the procedure is the same), check if a sequence of words is a valid SATOR square, etc.

And by showing this, a few things become evident:

• That anyone claiming we're a step away from AGI is a goddamn liar, if not worse (a gullible pile of rubbish).
• That all talk about "hallucinations" is a red herring analogy.
• That the output of those models cannot be used in any situation where reliability is essential.

why do everybody want to make it count letters and stuff like this?

We do not applaud the tenor for clearing his throat, as they say; yes. But we also do not applaud the tenor who can't even do that.

This particular model was advertised as being able to use tools, like calculators. And it includes a calculator tool in the package.

So, it should in theory, not have the probabilistic limitations of its native algorithms.

However, there are still the limitations created by multiple posts, where the model will change its answer in relation to previous inputs by users. This may have become even worse in model 5 because it supposedly remembers pervious conversations.