Technology
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
Let the record show that the most sophisticated LLM in the world is ultimately just a less competent version of Yes Man from New Vegas. And even Yes Man knew his programmer was stupid for designing him that way.
So their chatbot is able to change the email address used to recover an account? I guess, they vibe coded that system.
Even hacking is an AI-backed service nowadays...
Vibe hacking
Hacking before: Pull up hood on hoodie, open laptop, open terminal, type in a bunch of matrix code, bam "were in"
Hacking now: "Hack into this thing for me" No! "Pretty please?" Access granted!
Hacking by social engineering has always been far more common than hacking by exploiting code vulnerabilities.
alias prettyPlease="sudo"
Did the chatbot just send the recovery code to a Telegram channel?!? (Picture of phone with broken display)
So is Gemini the only one of these things competently designed?
I wouldn't count on it.
Securing these things is a freaking nightmare.
Giving the AI authority is what makes it powerful, it can do what an army of customer service agents can't.
But keeping it reigned in then becomes the same exact level of problem.
The best thing you can do is make tooling with protection and make the AI only use the tooling,
Just don’t allow it to do any administrative access.
How on earth did you come to that conclusion from this article
What have they done right?
Nothing. If it’s Google operated it’s probably full of issues. They are in the process of merging Gemini into their search engine, probably because not enough people are using it and they need to force it on people. Likewise for other chat bots from other companies.
Why would the LLM tool have access to send recovery emails to non account verified emails at all?
That’s insane.
Who else is going to have access to it when you keep laying off all the people?
Because one of the biggest companies on the planet that has issues with account takeovers clearly has no internal red team working on this stuff.
I guarantee they do have a red team that most likely flagged this as an obvious and severe risk. It was ignored by suits experiencing AI psychosis.
Because AI bros are incredibly deluded about both the capability of AI, and by extension their own capabilities using AI>
This isn't even a hack, it's just poorly written endpoints.
Would you consider phreaking equivalent to hacking? This is AI phreaking.
TIL about phreaking
Then this is also probably new to you
I need to set aside some time to read that although I'm not an anarchist myself.
It was largely overblown due to it getting banned. It was also published in the height of the Vietnam War, when the big evil communists were coming to brainwash your children into eating each other. It has a lot of blatantly incorrect info, which could be outright “blow up in your face” dangerous to anyone attempting the things in it. It’s not all wrong, but certain recipes have incorrect info that could easily lead to accidents.
Also fair warning, the UK will give people hard prison time simply for owning it. So maybe keep that shit onion-encrypted if you’re in the UK.
I linked to the Wikipedia article, not the handbook inself. And more for the (obsolete) phreaking content than the (highly dangerous) explosive content.
Kinda.
If you designed a publicly addressable system since 1985 and didn’t design it for security then you’re asking for it.
It's not phreaking. Social engineering.
The entity being manipulated is not human so I would not classify it as social engineering, even if similar techniques are used (help me my grandmother needs info).
should’ve asked it to delete the database instead, why else would it have that level of permissions.
Heh. Watched an old episode of Scorpion yesterday. The one with the armed hostage-takers who just had the one demand to the social media data mining company, to delete all the data they've mined. I amused myself a lot, by uttering "I like these guys".
Little Tommy Drop Tables.
I remember playing with the Gandalf security AI showcase/game and every 30 or so prompts, it would spit out massive amounts of raw training data or dev directives. AI just isn’t there yet. If you’re using it for sensitive topics, I’m losing respect for you. There is no gray area. You are an idiot if you give your AI this level of access.
It's not just not there yet. This is almost certainly not going the right direction to ever be "there" if there is something that can handle security issues. It's just not the right tool for the job, and I can't understand how so much of our economy is just assuming it is the right tool for every job.
Surely it will get there if we build enough datacenters?
Another banger from 404 media. This made my day.
I had always heard that 99% of hacking is just social engineering. AI has made that 100%.