Prontomomo

joined 2 years ago
[–] Prontomomo@lemmy.world 5 points 4 months ago

I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling