I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

you are viewing a single comment's thread
view the rest of the comments

[–] INeedMana@piefed.zip 3 points 2 days ago (1 children)

https://en.wikipedia.org/wiki/Slashdot_effect but people were going in, looking at the link, sometimes overwhelming the server

[–] artyom@piefed.social 3 points 1 day ago (1 children)

That's not at all the same thing. That's just the inability to handle large amounts of unexpected normal user traffic.

This is essentially bot traffic driven by servers all independently but simultaneously fetching the same image, without even a single human necessarily looking at it.

[–] INeedMana@piefed.zip 5 points 1 day ago (2 children)

From server's perspective the only difference between those is the user-agent. Traffic is the same

[–] HereIAm@lemmy.world 5 points 1 day ago (1 children)

But what OP is talking about is a technical detail that can be fixed.

[–] INeedMana@piefed.zip 3 points 1 day ago* (last edited 1 day ago)

Yes, but then a comment mentioned the "hug of death" - high spike of requests resulting from a link being shared on some site, and OP asked why centralized sites were also creating problems

There was another post regarding previews some months/(knowing me probably a year) ago and personally I would go with federating previews. With a button for mods to refresh it from source for when something goes fucky

But social networks have been creating random bursts of requests, this way or another, even in the times of centralized socials

[–] artyom@piefed.social 1 points 1 day ago

The origin and purpose of the traffic is not the same. That's why they block one and not the other.