Comment on Google Is the Only Search Engine That Works on Reddit Now Thanks to AI Deal

• Reverendender@sh.itjust.works ⁨3⁩ ⁨months⁩ ago

  Are they though?

  source

  • Evotech@lemmy.world ⁨3⁩ ⁨months⁩ ago

    I don’t know of any law that says that they can’t.

    source

    • reddig33@lemmy.world ⁨3⁩ ⁨months⁩ ago

      I also don’t know if a law that says search engines have to honor a robots.txt file. I guess we will see what happens if Bing r someone else decides to ignore it.

      source

      • Evotech@lemmy.world ⁨3⁩ ⁨months⁩ ago

        You can just require a log in to view content, or just flat out auto ban indexing robots.

        source
• helenslunch@feddit.nl ⁨3⁩ ⁨months⁩ ago

  How can they do that, logistically?

  Like I realize there’s a flag they can raise that asks not to be indexed but that’s not necessarily a legal requirement.

  source

  • Evotech@lemmy.world ⁨3⁩ ⁨months⁩ ago

    I guess they can make it hard to index by scraping by rate limiting or requiring login to view content etc and only provide Google the api to bypass the restrictions

    There’s probably a lot of ways to do it

    source