When sites regularly have a robots.txt that says “only Google may crawl”
Is that actually true?
If so, why would they do that?
Comment on AI companies are violating a basic social contract of the web and and ignoring robots.txt
lily33@lemm.ee 9 months ago
What social contract? When sites regularly have a robots.txt
that says “only Google may crawl”, and are effectively helping enforce a monolopy, that’s not a social contract I’d ever agree to.
When sites regularly have a robots.txt that says “only Google may crawl”
Is that actually true?
If so, why would they do that?
Imgonnatrythis@sh.itjust.works 9 months ago
I had a one-eared rabbit. He was a monolopy.
Ashelyn@lemmy.blahaj.zone 9 months ago
Sounds like a Pal name lol
assassin_aragorn@lemmy.world 9 months ago
Only if its model is a Lopunny missing an ear
Ashelyn@lemmy.blahaj.zone 9 months ago
I was thinking of a short lil bunny wearing a top hat and monocle with one ear sticking out of the center of the top hat but that works too