Comment on Perplexity AI is complaining their plagiarism bot machine cannot bypass Cloudflare's firewall

Kissaki@feddit.org ⁨2⁩ ⁨days⁩ ago

• lime@feddit.nu ⁨2⁩ ⁨days⁩ ago

  yeah it’s almost like there as already a system for this in place

  source

  • seraphine@lemmy.blahaj.zone ⁨1⁩ ⁨day⁩ ago

    THE CAKE DAY IS NOW. (i dont have an image at hand)

    source

    • lime@feddit.nu ⁨1⁩ ⁨day⁩ ago

      i really wish we wouldn’t do those. feels too reddity.

      source

      • seraphine@lemmy.blahaj.zone ⁨1⁩ ⁨day⁩ ago

        as you wish

        source

        • -> View More Comments
• ubergeek@lemmy.today ⁨2⁩ ⁨days⁩ ago

  And I’m assuming if the robots.txt state their UserAgent isn’t allowed to crawl, it obeys it, right? :P

  source

  • Kissaki@feddit.org ⁨1⁩ ⁨day⁩ ago

    No, as per the article, their argumentation is that they are not web crawlers generating an index, they are user-action-triggered agents working live for the user.

    source

    • ubergeek@lemmy.today ⁨1⁩ ⁨day⁩ ago

      Except, it’s not a live user hitting 10 sights all the same time, trying to crawl the entire site… Live users cannot do that.

      That said, if my robots.txt forbids them from hitting my site, as a proxy, they obey that, right?

      source
• drmoose@lemmy.world ⁨2⁩ ⁨days⁩ ago

  Its not up to the hoster to decide whom to serve content. Web is intended to be user agent agnostic.

  source