Technology

I’m willing to bet you could convince a subset of those people that such an LLM is in fact the second coming of Christ. So not just some tool “approved” by God, but that the LLM is the God itself.

Then (to the “true believers” minds) whatever it says will be unquestionable. And then whomever is pulling the strings behind the scenes can commit whatever atrocities they desire.

I’m familiar with TempleOS — but it really doesn’t have any applicability here. It’s just something written by a guy with some mental illness who thought God was telling him what he wanted in an Operating System. But even for the faithful it’s just a tool — like how a temple itself may be an important holy place, but isn’t itself worshipped by the people who use it. Nobody considers a church to actually be their God.

That’s vastly different from an LLM that purports to be itself divine. We can setup an LLM that actually claims to be the second coming of Jesus, and there will be people will do whatever it tells them to because of belief. If you suck in enough people for enough years slowly enough to build up a cult following, and abuse them just enough to keep them in line, you’ll be able to tell them to do all sorts of truly atrocious things — and some subset will in fact go through with them.

And yes, people can do that already (see Jim Jones, David Koresh, or any other cult leader that convinced all their followers to kill themselves and their families) — but an LLM could have a vastly larger reach around the globe. We may not need for the LLM itself to become Skynet — one or two bad actors behind the scenes of a “divine” LLM might be enough to bring down humanity all by itself.