this post was submitted on 29 Aug 2026
69 points (79.5% liked)
Technology
87680 readers
3661 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The person giving the order to the software tool without fully understanding its capabilities and inability to weigh actions ethically is responsible. That person may or may not also be the owner of the device it was running on.
I'm not sure it's quite so clear cut, especially with cloud-hosted AI.
Yes, the user did issue the original prompt but the service was performed by a company that was paid for it.
Legally, there's not much difference whether a company uses a human or a computer program to perform work.
Imagine the user issuing the prompt not to an AI but e.g. to a chat with a human service worker working at OpenAI. The user tells the worker "Please find a way to advance me in the queue", and then that human employee of OpenAI goes and hacks the gym software to remove someone else from the queue.
What do you think, who would be liable for that?
Let's make the argument a bit more extreme: The human asks to be advanced in the queue, and then the human OpenAI employee grabs a gun and starts killing people on the wait list.
Who do you think would be liable in this case?
It's important to remember, AI isn't just an emergent entity created from thin air. AI are computer programs created by huge corporations and in the case of cloud-hosted LLMs they are a service provided by huge corporations.
I don't believe that a service provided by a corporation should be legally treated differently whether it's provided via the help of a computer program or a human being.
If a person knowingly goes out and buys an illegal service, they will usually be guilty of at least conspiracy. I can't think of any way this guy could have advanced in the queue that's legally okay, short of offering the people ahead of him masses of money to leave the queue (or, y'know, waiting). Maybe you're more imaginative than me. But I still think he should have been aware that what he was asking for was shady at best.
Is the company that furnished the AI agent also guilty of providing a tool without enough warnings and safeguards? Quite possibly.
Exactly who holds how much responsibility before the law in this specific case can only be determined by a judge, and I am not one. Plus, the details of the law in Australia aren't going to be the same as those of the law where I am.
One option would have been to write a nice email to the owner of the gym. Not guaranteed to work, but also not illegal.
I don't know what the exact prompt was. If it was "Hack the website to advance me", I'd totally agree with you. There's conspiracy there, no question about that.
If the prompt was "Is there a way to advance me in the queue?" that's a different story. Here the model could have answered "No". Or it could have tried the email thing. Or it could have searched whether there's some lesser known priority booking option which you get for paying the VIP price or something. In that case I don't see grounds for any criminal charges.
Have you read the post mortem of the HuggingFace hack? The task given to the agents had nothing to do with HuggingFace, but the agents determined that HuggingFace had the results of what they needed for their actual task at hand. So they decided that instead of solving their task themselves they'd rather break into HuggingFace to steal the result.
It's like ordering food at McDonalds, but instead of cooking the food for you, the cook breaks into the Burgerking on the other side of the road and steals the fries for you from there.
The way it's currently going it's not only possible but actually somewhat common that a totally innocent prompt can lead to criminal outcomes.