this post was submitted on 20 Oct 2026
682 points (98.4% liked)

Fediverse

44460 readers
547 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, Mbin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)

founded 3 years ago
MODERATORS
 

I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

(page 2) 50 comments
sorted by: hot top controversial new old
[–] rimu@piefed.social 78 points 2 days ago (2 children)

Lemmy does this to itself, too - when you create a post it federates out and then every Lemmy instance that receives it makes a GET request to retrieve the post from the original server. Even though it just received the exact same data. Some other fedi software is even worse - Akkoma does a GET of the id of every activity it receives, not just for posts but for Likes too.

It's insane.

Anyway, proper caching at the reverse proxy / Cloudflare level solves it. The same kind of caching setup would do the same for sites serving HTML, like NPR too.

[–] Ghoelian@piefed.social 6 points 1 day ago (2 children)

That's weird. Is that how it's supposed to work, or did they just implement it in a bad way?

[–] The_Decryptor@aussie.zone 14 points 1 day ago

It's an access control method, called "authorized fetch". Rather than broadcasting the data you just announce the source of it, and then any client that wants to show it needs to negotiate with the originating server to request the data (And then blocklists and such come into play)

Say you've got 3 users, A, B, and C. A has blocked C, B follows A, and C follows B. Now A makes a post, B sees it in their feed and reboosts it, what does C see?

If B's server sends the full response along, then C can see the contents of A's post even though they've been blocked by A. But if all it sends is the URL of the post, C's server needs to make a request for it (providing authentication), and then A's server can reject it, so C sees nothing.

[–] rimu@piefed.social 3 points 1 day ago

I'm not sure but my guess it's an attempt to deal with the problem where sometimes federation activities arrive in the wrong order (especially after an instance has been offline for a while) - by making a GET to the origin server you can be sure you've got the latest version of the content. There are probably better ways to achieve that, though.

PieFed prioritises efficiency over correctness so it just accepts whatever is sent to it. In 3 years no one has complained about seeing stale content although I'm sure it happens occasionally.

load more comments (1 replies)
[–] glitchdx@lemmy.world 7 points 1 day ago

So linking my own website here is a bad idea? Glad I haven't done that yet.

[–] KeenFlame@feddit.nu -3 points 18 hours ago (5 children)

I'm not exactly sure why a web server can't handle legit web traffic, just because it's a lot? I agree that smaller actors should be given a little breastfeeding but solving it on the other side is bad architecture. A legit request can never be compared to a a ddos attack, the overloaded net has to throttle and then the instances need to retry with a generous enough delay.

Options for solutions to another systems problems are all going to introduce more problems. Sure, the requests could honestly be routed to a cache and it would help this problem, att least until it breaks or is exploited. But it would never be truly solved that way.

load more comments (5 replies)
[–] ace_garp@lemmy.world 20 points 1 day ago* (last edited 1 day ago) (2 children)

Randomly stagger the times of each instance's request.

+1-300 seconds

[–] SkaveRat@discuss.tchncs.de 13 points 1 day ago

That's basically what's now implemented, iirc. At least in masto

[–] x0x7@lemmy.world 3 points 1 day ago

300 seconds is a lot.

[–] TropicalDingdong@lemmy.world 26 points 2 days ago (5 children)
[–] expatriado@lemmy.world 21 points 2 days ago (1 children)

Norwalk Public Radio. From Norwalk, Iowa.

load more comments (1 replies)
load more comments (4 replies)
[–] Die4Ever@retrolemmy.com 20 points 2 days ago (2 children)

they could include the link preview data as part of the federation

load more comments (2 replies)
[–] Pamasich@kbin.earth 10 points 1 day ago (9 children)

I don't really get it. The image is federated, there's no need for instances to refetch it. Just checked out some posts and indeed it's there, in the image property. With the URL pointing to the Lemmy instance.

So if this is actually an issue, that's just bad programming and needs to be fixed.

~~Edit: didn't notice OP uses Piefed. Unlike Lemmy and Mbin, it seems like Piefed doesn't federate it.~~

Edit: actually I'm blind, Piefed does federate it too, sorry piefed

load more comments (9 replies)
[–] INeedMana@piefed.zip 8 points 1 day ago

Also https:// itsfoss.com/news/mastodon-link-problem/

[–] frongt@lemmy.zip 12 points 2 days ago (11 children)

A slight delay would be perfectly sufficient.

load more comments (11 replies)
[–] LibertyLizard@slrpnk.net 11 points 2 days ago (1 children)

Is this why the automated title retrieval never works for NPR?

load more comments (1 replies)
load more comments
view more: ‹ prev next ›