Ooh, nice.
Comment on FediDB has stoped crawling until they get robots.txt support
swizzlestick@lemmy.zip 1 year agoIt would be better if the server could detect bots and send them down a rabbit hole
Already possible: Nepenthes.
Skepticpunk@lemmy.world 1 year ago
Semi_Hemi_Demigod@lemmy.world 1 year ago
ANY SITE THIS SOFTWARE IS APPLIED TO WILL LIKELY DISAPPEAR FROM ALL SEARCH RESULTS.
I’m sold
merthyr1831@lemmy.ml 1 year ago
that website feels like uncovering a piece of ancient alien weaponry