this post was submitted on 20 Oct 2025
653 points (98.4% liked)

Technology

76299 readers
2923 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
 
top 50 comments
sorted by: hot top controversial new old
[–] cheers_queers@lemmy.zip 6 points 1 day ago

even the websites for local health systems have been going down. At this point if we have an event like Y2K was supposed to be, we are so fucked lol

[–] magnolia_mayhem@lemmy.world 28 points 2 days ago (11 children)

Hey! Great idea! Let's make all of the internet rely on one service from one evil company! What could go wrong?

[–] grahamja@reddthat.com 8 points 2 days ago

Typical Amazon undersells everyone else to remove competition. Then after everyone has gone under reminds you it is a terrible idea. Hopefully we see businesses get burnt more and determine it is better to self host, or even better yet not need half their services on "the cloud" anyways.

load more comments (10 replies)
[–] mesamunefire@piefed.social 139 points 3 days ago* (last edited 3 days ago) (3 children)

Amazon sneezes, the whole internet catches a cold.

Im still seeing services up/down the entire day at work. Services that are not even AWS like Azure are slow for some reason (probably businesses failing over to other infa). Its crazy.

None of our in office infa is having issues. Managers are talking about fail-overs all day lol.

[–] tomiant@piefed.social 85 points 3 days ago (1 children)

When a handful of people own all the companies in the world, the whole world becomes a single point of failure.

load more comments (1 replies)
[–] Boozilla@lemmy.world 29 points 3 days ago (7 children)

There are some downstream / knock on effects going on which can be explained...but I can't help but wonder if today's story is bigger than just AWS. AWS saying it was an outage of a "few hours" for DynamoDB and DNS...and that doesn't line up all that great with what people are reporting in the wild . I'm not trying to start a conspiracy theory, just wondering what the post mortems will tell us, if anything. Obviously the suits want to keep embarrassing fuckups downplayed as much as possible.

[–] Carrot@lemmy.today 1 points 1 day ago

As someone with a little insider baseball knowledge, it was just a few hours down of DynamoDB and DNS. However, that caused EC2 to go down for ~1 day, which causes pretty much 1/3 of the internet to go down. Once EC2 sorts themselves out, then teams/companies (almost all amazon services use EC2 in the back end) that use EC2 have to get their ducks back in a row, and that can take any span of time, depending on how well their code was written to handle failures + how many people they are willing to pay oncall/overtime.

load more comments (6 replies)
load more comments (1 replies)
[–] HertzDentalBar@lemmy.blahaj.zone 118 points 3 days ago (2 children)

It's almost as if centralizing everything is fucking stupid.

[–] scarabic@lemmy.world 1 points 1 day ago (1 children)

At my work there was a big push to get things out of our own isolated data centers and into AWS because we had some services that were really very vulnerable and not equipped to serve a global distributed audience well. AWS solved that problem for us and our devs liked working with it. However now our AWS bill is so high that we’re pushing everyone to reduce it. I guess that makes sense. We’re not going back to self-hosting, just optimizing and reducing. But it sure feels like whiplash from “get everything into AWS” to “we have too much in AWS.”

[–] HertzDentalBar@lemmy.blahaj.zone 1 points 1 day ago (1 children)

So it sounds like rather than investing in your own systems you outsourced and now it costs far more.

Shit son that's the same as almost every industry.

[–] scarabic@lemmy.world 1 points 1 day ago* (last edited 1 day ago)

I don’t know if it costs far more but AWS is one big bill whereas running your own systems is a bunch of smaller expenses including real estate and employees in addition to hardware and utilities costs, insurance, etc which are harder to count. We see AWS on one big bill so the sticker shock is more palpable.

Moving to AWS didn’t just replace our services, it gave us a higher level of service reliability and availability. We would have had to double our spend to serve the globe as we really need to. AWS was the better option.

I think it costs a ridiculous amount because we moved over services that were built on self-run data centers where chron jobs were not billed on CPU cycles in a transparent way. Now they are. We can probably cut the AWS bill by as much as half through intelligent refactoring and optimization. Smarter data retention periods, killing off shit that isn’t worth it etc.

[–] innermachine@lemmy.world 21 points 3 days ago (1 children)

But the bean counters said it was the best idea!

[–] cmbabul@slrpnk.net 5 points 2 days ago

Accountants shouldn’t be allowed to make decisions beyond their personal lives. MBAs even more so

[–] SCmSTR@lemmy.blahaj.zone 6 points 2 days ago (2 children)

What are these random fucking websites though

I mean those are giant websites and organizations that exist in the mainstream, instructure is used by most schools nowadays

[–] dependencyinjection@discuss.tchncs.de 1 points 1 day ago (1 children)

Which is random? I’ve only not heard of three: Instructure, fanduel, and chime.

All the ones I’ve heard of are quite massive companies and I would assume the others are too.

[–] JackbyDev@programming.dev 1 points 1 day ago (1 children)
[–] dependencyinjection@discuss.tchncs.de 2 points 1 day ago (1 children)

Well I can’t be sad that that predatory industry took a hit. Thanks for letting me know who they were. Seems betting has gotten crazy in America when I thought it wasn’t a thing outside of certain places like Atlantic city or Vegas.

[–] JackbyDev@programming.dev 1 points 1 day ago

Yeah, sports betting is getting legalized in more and more US states. It's pretty crazy. During most live sports broadcasts there are tons of commercials for it and even on the stadiums there are ads on the walls.

[–] gedaliyah@lemmy.world 59 points 3 days ago

Maybe it was a bad idea for society to put 90% of the internet on one company's infrastructure.

[–] Matriks404@lemmy.world 12 points 2 days ago* (last edited 2 days ago) (2 children)

I am grateful that my past me (from few months ago) decided to study Slavic philology, instead of getting stupid tech job. I hate modern tech, including anything from big corporations.

I am also glad that I've switched to Linux (Debian ftw) to escape software enshittification.

load more comments (2 replies)
[–] Rooty@lemmy.world 48 points 3 days ago

Let's take a decentralized network and centalize it! Nothing will surely go wrong.

[–] Bishma@discuss.tchncs.de 75 points 3 days ago (4 children)

So Roblox is the only of those companies that can handle proper region failover if us-east-1 shits the bed? You young ops have gotten soft! Learn to live by ChaosMonkey or die by the Gorilla.

[–] NaibofTabr@infosec.pub 52 points 3 days ago (3 children)

Yeah, but proper failover and recovery requires additional infrastructure, and that costs money.

Hopefully a bunch of risk management people are writing I-told-you-so emails to C-suites right now.

[–] homesweethomeMrL@lemmy.world 41 points 3 days ago (1 children)

Yeah but not to worry, C-suites have pretty good filtering rules in place to never read them. Saves time, really.

load more comments (1 replies)
load more comments (2 replies)
load more comments (3 replies)
[–] darvocet@infosec.pub 66 points 3 days ago (8 children)

Jesus Amazon is a big enough company they should be doing their own hosting not using AWS.

[–] tomiant@piefed.social 56 points 3 days ago (1 children)

They're big enough to be a utility and be nationalized!

load more comments (1 replies)
load more comments (7 replies)
[–] SlartyBartFast@sh.itjust.works 1 points 1 day ago (2 children)

Are any lemmy instances run on aws? Been getting a lot less articles last couple days

[–] Typotyper@sh.itjust.works 6 points 1 day ago

Maybe the bots are run by AWS

[–] JackbyDev@programming.dev 1 points 1 day ago

I'd be surprised if none are run in AWS.

[–] peoplebeproblems@midwest.social 46 points 3 days ago

It's kind of telling when Amazon services like Ring, Prime Video, and Alexa failed over pretty quickly, but everything else just didnt.

There's no conspiracy here. It's just highlighting that Amazon could prepare for AWS outages in a region, but since everything auth related was on us-east-1, everything else went down.

Shortcuts are costly.

[–] thatonecoder@lemmy.ca 15 points 2 days ago (1 children)

…and I did not even notice it, aside from the news here.

[–] Rentlar@lemmy.ca 7 points 2 days ago

It really shows what a great life that we on the Fediverse are living, by making our tech independent to a degree from the largest tech providers.

[–] PhobosAnomaly@feddit.uk 28 points 3 days ago (4 children)
load more comments (4 replies)
load more comments
view more: next ›