This is a most excellent place for technology news and articles.
They're framing it as "AI haters" instead of what it actually is, which is people who do not like that robots have been programmed to completely ignore the robots.txt files on a website.
No AI system in the world would get stuck in this if it simply obeyed the robots.txt files.
The disingenuous phrasing is like "pro life" instead of what it is, "anti-choice"
Notice how it's "AI haters" and not "people trying to protect their IP" as it would be if it were say...China instead of AI companies stealing the IP.
I hope it's effective.
It's not. If it was, every search engine out there would be belly up at the first nested link.
Google/Bing just consume their own crawling traffic. You don't want to NOT show up in search queries right?
You don’t want to NOT show up in search queries right?
At this point?
I am fully ok NOT being in search engines for any of my sites. Organic traffic has always been much more valuable than inorganic traffic.
It's unclear how much damage tarpits or other AI attacks can ultimately do. Last May, Laxmi Korada, Microsoft's director of partner technology, published a report detailing how leading AI companies were coping with poisoning, one of the earliest AI defense tactics deployed. He noted that all companies have developed poisoning countermeasures, while OpenAI "has been quite vigilant" and excels at detecting the "first signs of data poisoning attempts."
Despite these efforts, he concluded that data poisoning was "a serious threat to machine learning models." And in 2025, tarpitting represents a new threat, potentially increasing the costs of fresh data at a moment when AI companies are heavily investing and competing to innovate quickly while rarely turning significant profits.
"A link to a Nepenthes location from your site will flood out valid URLs within your site's domain name, making it unlikely the crawler will access real content," a Nepenthes explainer reads.
It might be initially, but they'll figure out a way around it soon enough.
Remember those articles about "poisoning" images? Didn't get very far on that either
The poisoned images work very well. We just haven't hit the problem yet, because a) not many people are poisoning their images yet and b) training data sets were cut off at 2021, before poison pills were created.
But, the easy way to get around this is to respect web standards, like robots.txt
The way to get around it is respecting robots.txt lol
So instead of the AI wasting your resources and money by ignoring your robots.txt, you're going to waste your own resources and money by inviting them to increase their load on your server, but make it permanent and nonstop. Brilliant. Hey, even better, you should host your site on something that charges you based on usage, that'll really show the AI makers who is boss. 🤣
It's already permanent and nonstop. They're known to ignore robots.txt, and remove user agent on detection.
And the goal is not only to prevent resource abuse, but break a predatory model.
But, feel free to continue gracefully doing nothing while other takes action, it's bound to help eventually.
Hey, you don’t need to convince me, you’ve clearly already committed to bravely sacrificing your own time and money in this valiant fight. Go get ‘em, tiger! I look forward to the articles about AI being stopped coming out any day now.
