Programming

using it wrong

You can't claim that it's being used wrong when it's being used as advertised.

AI lords are catapulting shovels-full of ~~shit~~ AI at the wall and hedging the economy on something sticking.

I don't think that you are really disagreeing with the article - I read it mostly as an invitation to broaden your view on what it means to use AI as it is today, with current benefits and harms, and stop trying to mentally divorce yourself of the consequences of said use by calling it just a tool, as if tools were fundamentally neutral, because they aren't, as you said yourself.

I do have some gripes with your stance though.

Yes, we have local models. But have you ever tried to set up a coding assistant with a local model on consumer-grade hardware? LLMs that are small/performant enough to run on a laptop tend to suck really bad, compared to the cloud-based (data center driven) flagship models. You can barely use them for anything more advanced that "Write me a friendly email to my boss".

That's the whole problem - to approach anything like real "I can use this in production"-usefulness, you do need this gigantic mass of data centres. We are in the beginning stages of having that kind of usefulness, currently, with all its costs. But that doesn't mean we should keep doing it.

It does mean that using gigantic LLMs as an everything machine is an ill-conceived idea, because they are too fucking expensive to run measured by the kind of ressources we have on our planet, and while you can make them more efficient, it always comes at a cost to their usefulness (more hallucinations, less stringent logic in long tasks etc.).

I also want to question this idea of "Well, any bad consequences of LLM usage are just people using it wrong, it's not inherent", especially for "AI psychosis".

Most reports of "AI psychosis" I've read started out as something totally benign - someone had a random, unusual idea and they decided to talk to an LLM about it to get some feedback. You know, to not have any women in their life do emotional labour for them, or something.

And now, the inherent tool design of LLMs becomes important. The fine-tuning on LLMs used as chatbots can be optimized for different things - like giving the impression of being purely informative, or to be responsive to emotional markers in the prompt, or to structure answers in a way that keeps users engaged, to encourage them to burn their token rate limit and purchase more. The latter is something nearly all commercial LLMs do to some degree.

Human psychology is not a secret science. You know what is a lot of fun for many humans, and keeps them engaged? Being told that they're right. And really clever. Have great observations and thoughts, and to offer to develop them further, together. A lot of people enjoy being flattered, even by a machine.

And this is fun, so why not explore further where this chat leads?

And the story goes on - the idea is expanded upon, the LLMs praising responses become grander and grander, the usage becomes longer, the user dives deeper and their divorce from reality becomes bigger, and at some point, no other human shares their reality anymore, and that is what psychosis, at its core, is.

Some snap back out of it in time, others get lost in it.

My point being: yes, maybe these harms result in a misuse of LLMs. But they are absolutely exacerbated by the way current implementations are designed. And then, the actual misuse is to have used an LLM for brain-storming at all, instead of talking to a human, that is fully prepared to tell you that your idea is stupid.

This harm, from my perspective, IS an inherent risk in the current implementation of the technology, and so are others. These systems are being designed that way. The misuses you decry are actively advertised for and are being promoted, because they are also lucrative.

No tool is neutral.

On a personal note: I personally don't like working with LLMs. I noticed that by doing it extensively in the last couple of months, especially for educational and coding purposes. It does something bad with my brain and with my learning and thinking, and it impairs me in the long run - if I do this a lot, I don't think I will be able to do things without an LLM at some point - classic skill decay. I don't want that. Other people might not think so, and rather feel empowered. That's fine to me, but I would encourage everyone to test themselves on that regularly.

And on a political note: The politics around "AI" are, to me, not especially new. It's just end-stage-capitalism, again. It's a finance sector bubble that will hurt us all when it bursts, again. It's the devaluation of human labour, again. It's surveillance and data collection and unjust wealth distribution and disinformation and societal atomization and environmental destruction again, but faster. The type of thing that has been happening before just keeps on happening, more. I do think that that kind of change (mostly in velocity) is something to oppose.

I'm maybe less optimistic but I agree we need to recognize that to some degree this tech is here to stay and we need to plan accordingly to reduce harm. Shaming people for their consumption only causes suffering. We need to grow more communities to be more resilient and help whoever we can.