yes but you can configure anubis to allow specific bots with botPolicy.yaml; the allow-good include already whitelists internet archive.
this post was submitted on 05 Aug 2026
52 points (96.4% liked)
Open Source
49061 readers
702 users here now
All about open source! Feel free to ask questions, and share news, and interesting stuff!
Useful Links
- Open Source Initiative
- Free Software Foundation
- Electronic Frontier Foundation
- Software Freedom Conservancy
- It's FOSS
- Android FOSS Apps Megathread
Rules
- Posts must be relevant to the open source ideology
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
- !libre_culture@lemmy.ml
- !libre_software@lemmy.ml
- !libre_hardware@lemmy.ml
- !linux@lemmy.ml
- !technology@lemmy.ml
Community icon from opensource.org, but we are not affiliated with them.
founded 7 years ago
MODERATORS
Can they please add "Dillo/3.2.0"?
look in the crawlers directory and you can add any allows you want with the proper user agent. :)
Nah, that was a joke; Dillo is basicslly a graphical Lynx (both have useragent-exceptions on some big "js-only" sites). And the Anubis dev is, for a while now, working on a static version, if you believe the error message. Though that would be hard to do, if at all possible.
ooh i'm sorry i didn't catch that <3
Is there anything AI doesn't ruin?
Well, AI forces people to rethink a broken economy.
Don't forget "intelligence"-as-a-service™®©!
Hi ChatGPT, I can't find a job due to the AI bubble. What should I do?