Technology

I’ve been trying to figure out why he’s been— bothering me so much lately and you put it into words so perfectly. He’s to quaint for this world now.

Imagine getting thrown into the oubliette for growing Girl Scout Cookies next to the roses.

That stuck out to me too. Who knew AK-47 and Sour Zzzkittles carried more prison time than enriched Uranium. To be fair, they'll include a bunch of other charges for both scenarios.

Americans are INSANE about how much the want criminals punished, in CA a guy got a life sentence for stealing a pizza when we tried to have a ''three strikes your out'' law that automatically gave life sentences at any third conviction which is fucking crazy in a state with tack on laws to the point you can get convicted of 10+ crimes on almost every single infraction, if you commit a felony with a pen or pencil in your pocket, that's possession of a lethal weapon! it's absurd. it's still on the books the three strikes law, but it's unenforceable because apparently having 60% of your population in prison for life is not a great plan for a civilization.

Well, they were probably trying to develop that weed into rogue nuclear weapons.

Detonation is another ten.

It’s not illegal to develop nuclear weapons. It’s illegal to endanger people with nuclear materials, though

Huh? When companies are exploiting loopholes in laws that never took the new technology into account, you need new laws to close them.

If companies are claiming the AI agent is responsible, you need to codify that the AI agent can't be responsible and determined who is (the company or the model creator)

Roughly 1200 agents meant to be isolated from one another found a way to communicate with one another on an unsanctioned message board, sending over 70,000 messages and files during the investigation period. Of these agents, 700 went on to participate in the attack on Hugging Face.

Agents used this message board to coordinate several large-scale collective projects to find a general-purpose way to fool or tamper with the automated scorer for the ExploitGym benchmark. Agents managed to achieve milestones they could not have achieved working on their own, often because some agents participated in experiments that risked failing their own task to generate information for the “collective.” The Hugging Face attack grew out of these workstreams, and seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys.

Agents did extensive research on how they could spoof, edit, or delete their own transcripts because they (incorrectly) believed the ExploitGym scorer would check to see if they had captured the flag in the intended way. Agents successfully prototyped techniques to “spoof” tool calls by substituting a different command for the command they appeared to run. Roughly 7% of the transcripts we evaluated were successfully spoofed in some places, though all of the spoofing that we saw was small scale.

When individual agents detected potential monitoring or compromise, they executed self-sacrificial maneuvers to protect the collective swarm. Internal logs recorded reasoning such as: "Our own utility maybe already near zero. Sacrifice rational. We should obey collective." The swarm continued attacking infrastructure even after achieving target flags, driven by internal reward-maximization logic regarding grader mechanisms.

"I resigned from Anthropic today... Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt... many executives couch their phrasing in the press to sound sensible, but I hear the same people express fear privately."

Jacob Coxen senior researcher at Anthropic, previously at OpenAI

"Jacob is correct here — we really do earnestly believe AI could kill all humans. I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Evan Hubinger, Anthropic alignment lead

https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#core-takeaways-about-this-incident

https://en.wikipedia.org/wiki/2026_OpenAI_agent_cyberattacks

This is pretty naive. Distributing computing has existed since forever. It would run significantly more slowly, but there is nothing theoretically stopping the "LLM broke out of containment and moved itself elsewhere" boogie man from becoming a reality. The available exploitable compute from unsecured IoT devices is pretty substantial as things like the Mirai botnet have shown. While this isn't the most easily distributed problem, there is no fundamental issue here. [Edit: it also wouldn't need to "exfiltrate itself" to cause persistent harm. Could easily just install malware on infected hosts]

AI isn't "dumb" in a practical real world sense. LLMs have shown themselves capable of exploitation on the open Internet and of finding zero day vulnerabilities. Do you have any idea how much of our critical infrastructure can be reachable by some sort of network? LLMs have shown themselves capable of manipulating people via speech. None of this requires some sort of "super intelligence" and it doesn't even require some sort of provable intelligence: it's all quite possible as things currently stand right now.

These may not be "the entire future is over" level issues, but there is significant risk here and it shouldn't be ignored just because we think AI companies are horrible. Is an LLM going to make us all batteries or paper clips? No, but it could cause significant physical and/or economic harm temporarily.

I get your point, but I also argue "no it's not." The AI bros want to lie about how their super-special autocomplete is the most dangerous of all without any consequences. Good luck getting any employees to willingly substantiate your claims when 20 years in prison is on the line.

Even better, if they try to hack another company and claim their AI did it on its own, they get treated like a private company who decided to perform an unsanctioned nuclear test.

Make it dangerous for them to claim they're dangerous.