technology

I tend to agree, but I suspect they want to rile everyone up, get Chinese AI products “banned” in the us as national security threats (both cheaper commercial products and open source efficient models you can run yourself), and as a side benefit make themselves seem too important to fail so that when they hit the cliff next year when all their compute agreements start coming online, the government will bail them out. I doubt any of the guys running things have any serious interest worrying about kill switches or achieving “alignment” (if that was ever even a possible thing, who knows; humans aren’t exactly “aligned” for the good of humanity, we’re gonna smoke the whole atmosphere for fun). They’ve already spent oodles lobbying to avoid “bad regulation” but in reality there isn’t any real regulation they’d accept unless it was purely fig leaf.

The alternative is that people like Dario Amedei actually believe that the industry can “pace itself” globally, which would mean he’s incredibly stupid/delusional or both. It would be absurd if they actually believed the stuff they write; implausible. I get that possibly Anthropic is a little religious/culty, and that could explain them, but they’re all supposed super rationalists. They all know that if there’s anything to actually mine out of this gold rush, it will be mined, and humanity doesn’t really do pre-emptive self-limiting; we’re not going to chose, as a global population, to not explore this avenue. Usually the bad thing has to happen, and be very very bad, and then we do some international taboo policing about it for a generation or two.

Deep down they think it can be managed and controlled. And they want to be the ones who control it.

In my opinion, they can’t win this way. Actually I guess I believe they can’t win anyway in this. They might get a carved out market of the US to themselves, but they don’t beat China on these terms—the entire rest of the world will adopt Chinese ai, or derivatives, and they will likely surpass our development ability. They would have to achieve the god-like systems they’re always threatening to make. And if they do make those, I rate it very low that they will both be super-intelligent, and also controllable by a gaggle of Bay Area tech people.

Who knows I guess… entirely possible it will go the other way, but seems like one way or the other we’ll be finding out in the next little while.