this post was submitted on 21 Aug 2025
223 points (89.7% liked)

Selfhosted

54041 readers
737 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

  7. No low-effort posts. This is subjective and will largely be determined by the community member reports.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

Some thoughts on how useful Anubis really is. Combined with comments I read elsewhere about scrapers starting to solve the challenges, I'm afraid Anubis will be outdated soon and we need something else.

you are viewing a single comment's thread
view the rest of the comments
[–] interdimensionalmeme@lemmy.ml -4 points 4 months ago (9 children)

You need to set you http serving process to a priority below the administrative processes (in the place where you are starting it, so assuming linux server that would be your init script or systemd service unit).

Actual crash causing reboot ? Do you have faulty ram maybe ? That's really not ever supposed to happen from anything happenning in userland. That's not AI, your stuff might be straight up broken.

Only thing that isn't broken that could reboot a server is a watchdog timer.

You server shouldn't crash, reboot or become unreachable from the admin interface even at 100% load and it shouldn't overheat either, temperatures should never exceed 80C no matter what you do, it's supposed to be impossible with thermal management, which all processors have had for decades.

[–] poVoq@slrpnk.net 2 points 4 months ago (8 children)

Great that this is all theoretical 🤷 My server hardware might not be the newest but it is definitly not broken.

And besides, what good is that you can still barely access the server through ssh, when the cpu is constantly maxed out and site visitors only get a timeout when trying to access the services?

I don't even get what you are trying to argue here. That the AI scraper DDOS isn't so bad because in theory it shouldn't crash the server? Are you even reading what you are writing yourself? 🤡

[–] interdimensionalmeme@lemmy.ml -4 points 4 months ago (5 children)

Even if your server is a cell phone from 2015, if it's operating correctly and the cpu is maxed out, that means it's fully utilized and services hundreds of megabits of information.

You've decided to let the entire world read from your server, that indiscriminatory policy is letting people you don't want getting your data, get your data and use your resources.

You want to correct that by making everyone that comes in solve a puzzle, therefore in some way degrading their access, it's not surprising that they're going to complain. The other day I had to wait over 30 second at an anubis puzzle page, when I know that the AI scrappers have no problem getting through, something on my computer, probably some anti-crypto mining protection is getting triggered by it and now I can't no-script the web either because of that thing and it can't even stop scrappers anyway !

So, anubis is going to be left behind, all the real users are, for years, going to be annoyed and have their entire internet degraded by it while the scrappers got that institutionally figured out in days.

If it's freely available public data then the solution isn't restricting access trying to play a futile arms race with the scrapper and throwing the real users to the dogs, it's to have standardized incremental efficient database dumps so the scrappers stop assuming every website is interoperability-hostile and scrape them. Let facebook and xitter fight the scrappers, let anyone trying to leverage public (and especially user contributed data) fight the scrappers.

[–] poVoq@slrpnk.net 3 points 4 months ago (1 children)

Aha, an apologist for AI scraper DDOS, why didn't you say so directly instead of wasting my time?

[–] interdimensionalmeme@lemmy.ml -3 points 4 months ago (1 children)

The ddos is caused by the gatekeeping, there was no such issue before the 2023 API wars, fork over the goods and nobody gets hurt, it's not complicated, you want to publish information to the public, don't scrunch it up behind diseased trackers and ad infested pages which burn you cpu cycles. Or just put it in a big tarball torrent, the web is turning into a cesspool, how long until our browsers don't even query websites at all but self-hosted crawler and search like searxng, at least then I won't be catching cooties from your javascript cryptomining bots embed into the pages !

[–] Deathray5@lemmynsfw.com 2 points 4 months ago

"fork over the goods and nobody gets hurt" mate you are not sounding like the good person here

load more comments (3 replies)
load more comments (5 replies)
load more comments (5 replies)