Comment

Comment on Perplexity AI is complaining their plagiarism bot machine cannot bypass Cloudflare's firewall

I actually agree with them

This feels like cloudflare trying to collect rent from both sides instead of doing what’s best for the website owners.

There is a problem with AI crawlers, but these technologies are essentially doing a search, fetching a several pages, scanning/summarizing them, then presenting the findings to the user.

I don’t really think that’s wrong, it’s just a faster version of rummaging through the SEO shit you do when you Google something.

source

Sort:hotnew top

drspod@lemmy.ml ⁨12⁩ ⁨hours⁩ ago
What’s best for the website owners is to have people actually visit and interact with their website. Blocking AI tools is consistent with that.

source
- panda_abyss@lemmy.ca ⁨11⁩ ⁨hours⁩ ago
  For a lot of AI search I actually end up reading the pages, so I don’t know hope much this stops that
  
  source
  - AstralPath@lemmy.ca ⁨11⁩ ⁨hours⁩ ago
    You’re the outlier, I promise. People are literally forfeiting their brains in favor of an LLM transplant hese days.
    
    source
    pennomi@lemmy.world ⁨11⁩ ⁨hours⁩ ago
    On the flip side, most websites are so ad-ridden these days a reader mode or other summary tool is almost required for normal browsing. Not saying that AI is the right move, but I can understand not wanting to visit the actual page any more.
    
    source
    -> View More Comments
kopasz7@sh.itjust.works ⁨11⁩ ⁨hours⁩ ago
Search engines been going relatively fine for decades now. But the crawlers from AI companies basically DDOS hosts in comparison, sending so many requests in such a short interval. Crawling dynamic links as well that are expensive to render compared to a static page, ignoring the robots.txt entirely, or even using it discover unlinked pages.

Servers have finite resources, especially self hosted sites, while AI companies have disproportinately more at their disposal, easily grinding other systems to a halt by overwhelming them with requests.

source
pr06lefs@lemmy.ml ⁨11⁩ ⁨hours⁩ ago
If a neighborhood is beset by roving bands of thieves, sooner or later strangers will be greeted by a shotgun rather than an invitation to tea, regardless of their intentions. Them’s the breaks. Bots are going to take a hit now and their operators are just going to have to deal with it. Sucks when people don’t play nice, but this is what you get.

source
- FauxLiving@lemmy.world ⁨10⁩ ⁨hours⁩ ago
  I’m sure people that are attempting to drive to their house in a new vehicle wouldn’t appreciate being riddled with bullets because the neighborhood watch makes no attempt to distinguish between thieves and homeowners.
  
  source
  - pr06lefs@lemmy.ml ⁨10⁩ ⁨hours⁩ ago
    So sad for them. Try not living in a war zone?
    
    source
    FauxLiving@lemmy.world ⁨10⁩ ⁨hours⁩ ago
    It isn’t a war zone, it’s a gated community where the guards have suddenly decided that any vehicle made after 2020 is full of thieves without bothering to consult the residents or giving them the ability to opt out of having their dinner guests murdered for driving a vehicle the security guards don’t like.
    
    source
    -> View More Comments