I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

you are viewing a single comment's thread
view the rest of the comments

[–] KeenFlame@feddit.nu -3 points 16 hours ago (1 children)

I'm not exactly sure why a web server can't handle legit web traffic, just because it's a lot? I agree that smaller actors should be given a little breastfeeding but solving it on the other side is bad architecture. A legit request can never be compared to a a ddos attack, the overloaded net has to throttle and then the instances need to retry with a generous enough delay.

Options for solutions to another systems problems are all going to introduce more problems. Sure, the requests could honestly be routed to a cache and it would help this problem, att least until it breaks or is exploited. But it would never be truly solved that way.

[–] artyom@piefed.social 6 points 16 hours ago (1 children)

I'm not exactly sure why a web server can't handle legit web traffic, just because it's a lot?

Its not because it's a lot, it's because it all happens in the same instant, which does not resemble any sort of normal web traffic.

but solving it on the other side is bad architecture

Solving it on the side that's creating the problem is bad architecture? 🤔

But it would never be truly solved that way.

Several potential solutions have been posited, if you just look through the comments.

[+] KeenFlame@feddit.nu -6 points 15 hours ago (1 children)

Lots of normal traffic is instant. Throttling must be handled by for in this instance NPR or its providers routers because even if hammering is a bad practice, we are participating in why it happens at all if we just build structures to mitigate it.

Yes that's why I included a solution, and explained explicitly that it can be solved but only temporarily

[–] artyom@piefed.social 1 points 15 hours ago (1 children)

Lots of normal traffic is instant

Not instant from a thousand different servers. That's what a botnet looks like. AKA DDOS. That's why I explained that. Why are you acting like you don't understand this?

Throttling must be handled by for in this instance NPR

No. It mustn't. AP servers can fix it on their end.

Yes that's why I included a solution

Which you then went on to explain wasn't a solution...?

[–] KeenFlame@feddit.nu -4 points 14 hours ago (1 children)

You may suffer from irritation of your ego my friend. Know that you are not your ideas or projects, you are more than that. I feel like all of your questions were previously answered sorry.

[–] artyom@piefed.social 1 points 13 hours ago

You may suffer from the dumbs, my friend. My questions were answered. Before you showed up.