Selfhosted

59999 readers

441 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

285

Anubis - Weighs the soul of incoming HTTP requests using proof-of-work to stop AI crawlers (github.com)

submitted 1 year ago* (last edited 1 year ago) by zutto@lemmy.fedi.zutto.fi to c/selfhosted@lemmy.world

68 comments fedilink hide all child comments

I just started using this myself, seems pretty great so far!

Clearly doesn't stop all AI crawlers, but a significantly large chunk of them.

you are viewing a single comment's thread
view the rest of the comments

[–] poVoq@slrpnk.net 21 points 1 year ago* (last edited 1 year ago) (1 children)

It requires a bunch of browser features that non-user browsers don't have, and the proof-of-work part is like the least relevant piece in this that only gets invoked once a week or so to generate a unique cookie.

I sometimes have the feeling that as soon as some crypto-currency related features are mentioned people shut off part of their brain. Either because they hate crypto-currencies or because crypto-currency scammers have trained them to only look at some technical implementation details and fail to see the larger picture that they are being scammed.

[–] swelter_spark@reddthat.com 2 points 1 year ago (1 children)

So if you try to access a website using this technology via terminal, what happens? The connection fails?

[–] crmsnbleyd@sopuli.xyz 3 points 1 year ago (1 children)

If your browser doesn't have a Mozilla user agent (I.e. like chrome or Firefox) it will pass directly. Most AI crawlers use these user agents to pretend to be human users

[–] swelter_spark@reddthat.com 1 points 1 year ago (1 children)

What I'm thinking about is more that in Linux, it's common to access URLs directly from the terminal for various purposes, instead of using a browser.

[–] crmsnbleyd@sopuli.xyz 1 points 1 year ago

If you're talking about something like curl, that also uses its own User agent unless asked to impersonate some other UA. If not, then maybe I can't help.