this post was submitted on 08 Nov 2026
304 points (91.3% liked)

Ask Lemmy

42304 readers
1498 users here now

A Fediverse community for open-ended, thought provoking questions


Rules: (interactive)


1) Be nice and; have funDoxxing, trolling, sealioning, racism, toxicity and dog-whistling are not welcomed in AskLemmy. Remember what your mother said: if you can't say something nice, don't say anything at all. In addition, the site-wide Lemmy.world terms of service also apply here. Please familiarize yourself with them


2) All posts must end with a '?'This is sort of like Jeopardy. Please phrase all post titles in the form of a proper question ending with ?


3) No spamPlease do not flood the community with nonsense. Actual suspected spammers will be banned on site. No astroturfing.


4) NSFW is okay, within reasonJust remember to tag posts with either a content warning or a [NSFW] tag. Overtly sexual posts are not allowed, please direct them to either !asklemmyafterdark@lemmy.world or !asklemmynsfw@lemmynsfw.com. NSFW comments should be restricted to posts tagged [NSFW].


5) This is not a support community.
It is not a place for 'how do I?', type questions. If you have any questions regarding the site itself or would like to report a community, please direct them to Lemmy.world Support or email info@lemmy.world. For other questions check our partnered communities list, or use the search function.


6) No US Politics.
Please don't post about current US Politics. If you need to do this, try !politicaldiscussion@lemmy.world or !uspolitics@lemmy.world


7) No Hit-and-Run questions.
Please don't delete your post for no apparent reason. If you plan on deleting a question later, say so in the post, or if you feel that you have a good reason to remove it, message a mod beforehand. It's not fair to the ones who took their time to answer, and it's not in the spirit of the community.


8) No Bots.
Posts or comments from bots, LLM's, AIs, Neural Networks, Transformers, or Marvin the Paranoid Android are not welcome in AskLemmy. Real humans only please.


Reminder: The terms of service apply here too.

Partnered Communities:

Tech Support

No Stupid Questions

You Should Know

Reddit

Jokes

Ask Ouija


Logo design credit goes to: tubbadu


founded 3 years ago
MODERATORS
 

I’m getting really tired of “verify you’re not a bot” checks everywhere.

I understand these checks help prevent bots, but they seem to appear on so many websites now. Does anyone else find them frustrating?

you are viewing a single comment's thread
view the rest of the comments
[–] mesamunefire@piefed.social 15 points 1 day ago* (last edited 1 day ago) (1 children)

I created a bot black hole. Specific recursive link that if the click it three times it blocks the IP. And the link is "click if ai" URL at the bottom. Works like a charm.

[–] GenosseFlosse@feddit.org 12 points 1 day ago (2 children)

No; the traffic i get seems like it had stored my sitemap from ~10 years ago because the requested urls there are long gone, and now tries to guess other url's with a similar pattern. It does not seem to follow links from the frontpage.

[–] Gullible@sh.itjust.works 8 points 1 day ago (1 children)

That's fascinating. It almost sounds like they're training a scraper. Which, really, is far and away the worst possible bot to have.

[–] FooBarrington@lemmy.world 3 points 1 day ago (2 children)

Training a scraper? Sounds more like they're just scraping training data.

[–] Gullible@sh.itjust.works 1 points 8 hours ago

You're entirely right, but I just wanted to post something innocuous to confirm that I picked up a stalker. Sorry for involving you!

[–] PapaStevesy@lemmy.world 2 points 21 hours ago* (last edited 21 hours ago) (1 children)

Maybe they're dating train scrapings

[–] Kangae_Hishiryo@scribe.disroot.org 1 points 19 hours ago* (last edited 19 hours ago)

They're straining craping dats

[–] mesamunefire@piefed.social 4 points 1 day ago (1 children)

I have fail2ban take care if those. 404s and all that.

[–] GenosseFlosse@feddit.org 7 points 1 day ago (1 children)

I already looked into something similar, IP banning does not work. One day i got ~120k unique IPs, but the most active one only made ~100 requests. I would gain nothing if i run any script to lookup the IP with each requests (and using up more resources in the process), if it fails 99% of the time because the next hit is a different IP.

[–] mesamunefire@piefed.social 4 points 1 day ago* (last edited 1 day ago) (1 children)

Ok. Sounds like your in a unique situation. Good luck. You seem fucked.

All the above worked for me.

[–] CorrectAlias@piefed.blahaj.zone 1 points 1 day ago* (last edited 1 day ago) (1 children)

I also have the same problem as who you're replying to on a few of my websites. The only way to block them is Anubis or Cloudflare WAF.

I think you might be in the unique situation, to be honest. It may only be a matter of time until your solution also stops working.

[–] grysbok@lemmy.sdf.org 1 points 20 hours ago

Agreed. The current situation seems to be a gazillion bots each making a single request. I've had some luck blocking whole ASNs (Alibaba, I'm glaring at you), and Anubis. I've had pretty good uptime the last few months.