Comment on I was wrong about robots.txt

• General_Effort@lemmy.world ⁨1⁩ ⁨day⁩ ago

  It would be a lot to write, if you had to say what something does not do rather than what it does.

  I looked at what the Cloudflare CEO said again. To be fair to him, he is not actually backing you up. He’s saying that Google makes no difference between the AI overview and the other search results. That is true. The AI overview is a search feature. I’m not sure why someone would want their link listed in search but not appear much more prominently in the AI overview.

  source

  • ell1e@leminal.space ⁨1⁩ ⁨day⁩ ago

    But the article later does back it up: “Although Cloudflare singled out Google, other search engines that view AI search features as part of their search products also use the same bots for training as they do for search indexing.”

    source

    • General_Effort@lemmy.world ⁨1⁩ ⁨day⁩ ago

      But the article later does back it up

      The CEO of Cloudflare did not assert that. I was surprised that he would claim such a thing, and that should have made me read more carefully. Elon Musk notwithstanding, neither incompetence nor conspiracy theorizing are common at that level, publicly anyway.

      You can believe whatever you like, of course. Freedom of opinion is nothing if not the right to be wrong.

      source

      • ell1e@leminal.space ⁨1⁩ ⁨day⁩ ago

        Right, but the article does. Anyway, I got other things to do.

        source