ell1e
@ell1e@leminal.space
- Comment on I was wrong about robots.txt 17 hours ago:
Right, but the article does. Anyway, I got other things to do.
- Comment on Why are there so many german communities on Lemmy? 18 hours ago:
you made us proud!
- Comment on Why are there so many german communities on Lemmy? 19 hours ago:
you deserve a trophy 🏆 🥰
- Comment on Why are there so many german communities on Lemmy? 19 hours ago:
not that the government cares, they want to centralize most of the data of citizens now with pretty poor protections in a lot of cases. sads
- Comment on Why are there so many german communities on Lemmy? 19 hours ago:
i may or may not be german as well 🫣
- Comment on Why are there so many german communities on Lemmy? 20 hours ago:
interestingly, most commenters here don’t seem to be on .world 🤔
- Comment on Why are there so many german communities on Lemmy? 20 hours ago:
surprise germans 🫨
- Comment on I was wrong about robots.txt 21 hours ago:
But the article later does back it up: “Although Cloudflare singled out Google, other search engines that view AI search features as part of their search products also use the same bots for training as they do for search indexing.”
- Comment on I was wrong about robots.txt 1 day ago:
You look up what Googlebot does. No AI.
I disagree that it says that. The Cloudflare CEO seems to disagree as well.
- Comment on I was wrong about robots.txt 1 day ago:
Nothing on this page seems to contradict the article.
- Comment on I was wrong about robots.txt 1 day ago:
So what’s the quote from your documentation that backs up your claim?
- Comment on I was wrong about robots.txt 1 day ago:
And allowing the public crawler might also have it feed their AI: arstechnica.com/…/cloudflare-wants-google-to-chan…
- Comment on I was wrong about robots.txt 1 day ago:
- Comment on I was wrong about robots.txt 1 day ago:
Often it is, but the problem is platforms conflate things with the questionable AI scraping crawlers to blackmail websites into participating in feeding AI.
For example, Googlebot if enabled won’t just list you for search, but will also scrape your contents for Google’s AI. I imagine LinkedinBot, given it’s microsoft, will feed some other AI of theirs as well on top of the previews.
Until regulation steps in to require AI bots to separately ask for crawling permission, or to actually get a proper license for reuse of the contents, this situation isn’t going to improve.