Comment on How to combat large amounts of Ai scrapers
Sheldan@lemmy.world 1 month agoSome of them are at least honest and have it as a user agent.
Comment on How to combat large amounts of Ai scrapers
Sheldan@lemmy.world 1 month agoSome of them are at least honest and have it as a user agent.
krakenfury@lemmy.sdf.org 1 month ago
Is ignoring robots.txt considered “honest”?
Sheldan@lemmy.world 1 month ago
That’s not what I was talking about