this post was submitted on 30 Apr 2026
97 points (86.5% liked)

Technology

42839 readers
219 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:


This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 4 years ago
MODERATORS
 

It only took nine seconds for an AI coding agent gone rogue to delete a company’s entire production database and its backups, according to its founder. PocketOS, which sells software that car rental businesses rely on, descended into chaos after its databases were wiped, the company’s founder Jeremy Crane said.

The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, which is one of the AI industry’s flagship models. As more industries embrace AI in an attempt to automate tasks and even replace workers, the chaos at PocketOS is a reminder of what could go wrong.

Crane said customers of PocketOS’s car rental clients were left in a lurch when they arrived to pick up vehicles from businesses that no longer had access to software that managed reservations and vehicle assignments.

you are viewing a single comment's thread
view the rest of the comments
[–] Powderhorn@beehaw.org 40 points 2 days ago (4 children)

Why in the everliving fuck would you give software delete access to your live backups? Like, in what scenario is this a solution?

[–] ATS1312@lemmy.dbzer0.com 4 points 1 day ago

Bear in mind this same company had their "backups" on the same drive as production.

That tells you a LOT about who is formulating these "solutions"

[–] chicken@lemmy.dbzer0.com 32 points 2 days ago (1 children)

The trend seems to be to give an AI agent access to the same command line and credentials a person would use, with no sandboxing, because then it can do the same tasks in a similar way and "just works". Obviously this is insane, and not even attempting building a comprehensive sandboxing system to deploy an AI agent into invites disaster, but you can see why certain people would be tempted, because that would take a lot of work and thought and probably need a human in the loop in the end anyway.

[–] dfyx@lemmy.helios42.de 12 points 1 day ago (2 children)

Even a person should not be able to delete critical backups without jumping through a couple of hoops.

[–] Town@lemmy.zip 3 points 1 day ago

And critical backups should be passed into an air gapped vault with a little guard piggy.

[–] Swedneck@discuss.tchncs.de 3 points 1 day ago

it's the kind of thing that should literally require 3 people turning physical keys at the same location

[–] LukeZaz@beehaw.org 12 points 2 days ago

When you believe AI can do anything, you don't worry about what sorts of access it'll break things with. When you rely on AI to do work, you're too interested in half-assing your job to consider what might go wrong. When capitalism never promotes people for their skill, understanding or caution, the former two issues proliferate.

Voilà, disaster.

[–] JustJack23@slrpnk.net 2 points 1 day ago

That is their disaster recovery plan "ask Claude"