Comment on Cloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.

RelativeArea1@sh.itjust.works ⁨1⁩ ⁨year⁩ ago

• drmoose@lemmy.world ⁨1⁩ ⁨year⁩ ago

  Lol website traffic accounts for like 1% of bandwidth budget. 1 netflix movie is like 20k web pages.

  source
• dual_sport_dork@lemmy.world ⁨1⁩ ⁨year⁩ ago

  Especially since the solution I cooked up for my site was to identify the incoming requests from these damn bots – which is not difficult, since they ignore all directives and sanity and try to slam your site with like 200+ requests per second, that makes 'em easy to spot – and simply IP ban them.

  In fact, anybody who doesn’t exhibit a sane crawl rate gets blocked from my site automatically. For a while, most of them were coming from Russian IP address zones for some reason. These days Amazon is the worst offender, I guess their Rufus AI or whatever the fuck it is tries to pester other retail sites to “learn” about products rather than sticking to its own domain.

  Fuck 'em. Route those motherfuckers right to /dev/null.

  source

  • desktop_user@lemmy.blahaj.zone ⁨1⁩ ⁨year⁩ ago

    the only problem with that solution being applied to generic websites is schools and institutions can have many legitimate users from one IP address and many sites don’t want a chance to accidentally block one.

    source

    • dual_sport_dork@lemmy.world ⁨1⁩ ⁨year⁩ ago

      This is fair in those applications. I only run an ecommerce web site, though, so that doesn’t come into play.

      source
  • morrowind@lemmy.ml ⁨1⁩ ⁨year⁩ ago

    Cloudflare offers that too, but you can’t always tell

    source
  • Buelldozer@lemmy.today ⁨1⁩ ⁨year⁩ ago

    and try to slam your site with like 200+ requests per second

    Your solution would do nothing to stop the crawlers that are operating 10ish rps. There’s ones out there operating at a mere 2rps but when multiple companies are doing it at the same time 24x7x365 it adds up.

    Some incredibly talented people have been battling this since last year and your solution has been tried multiple times. It’s not effective in all instances and can require a LOT of manual intervention and SysAdmin time.

    thelibre.news/foss-infrastructure-is-under-attack…

    source

    • confusedbytheBasics@lemmy.world ⁨1⁩ ⁨year⁩ ago

      Yep. After you ban all the easy to spot ones you’re still left with far too many hard to ID bots. At least if your site is popular and large.

      source
    • dual_sport_dork@lemmy.world ⁨1⁩ ⁨year⁩ ago

      It’s worked alright for me. Your mileage may vary.

      If someone is scraping my site at a low crawl rate I honestly don’t care so long as it doesn’t impact my performance for everyone else. If I hosted anything that was not just public knowledge or copy regurgitated verbatim from the bumf provided by the vendors of the brands I sell, I might oppose to it ideologically. But I don’t. So I don’t.

      If parallel crawling from multiple organizations legitimately becomes a concern for us I will have to get more creative. But thus far it hasn’t, and honestly just wholesale blocking Amazon from our shit instantly solved 90% of the problem.

      source
  • Flagstaff@programming.dev ⁨1⁩ ⁨year⁩ ago

    Geez, that’s a lot of requests!

    source

    • dual_sport_dork@lemmy.world ⁨1⁩ ⁨year⁩ ago

      It sure is. Needless to say, I noticed it happening.

      source
• ininewcrow@lemmy.ca ⁨1⁩ ⁨year⁩ ago

  It’s what I’ve been saying about technology for the past decade or two … we’ve hit an upper limit to our technological development … that limit is on individual human greed where small groups of people or massively wealthy people hinder or delay any further development because they’re always trying to find ways to make money off it, prevent others from making money off it, monopolize an area or section of society … capitalism is literally our world’s bottleneck and it’s being choked off by an oddly shaped gold bar at this point.

  source
• melpomenesclevage@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

  nothing can be improved while capitalism exists; all improvement will be seized and used to oppress.

  source

  • QuarterSwede@lemmy.world ⁨1⁩ ⁨year⁩ ago

    The problem you aren’t recognizing is that, until humans are no longer driven by self preservation, there will always be oppression in any system. They all have and will continue to breakdown. It’s easy to blame capitalism but even socialist systems eventually cave under the weight of greed and power. We are the problem mon frère.

    source

    • melpomenesclevage@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

      there will always be complications, sure. PERFECT may not be achievable.

      but we can’t have GOOD, we can’t have DECENT, we can’t have OKAY until the last owner is strangled with the entrails of the last enforcer.

      source
  • morrowind@lemmy.ml ⁨1⁩ ⁨year⁩ ago

    How can authority not exist? That’s staggeringly broad

    source

    • melpomenesclevage@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

      given what domains we’re hosted on; i think we’ve both had a version of this conversation about a thousand times, and both ended up where we ended up. do you want us to explain hypothetically-at-but-mostly-past each other again? I can do it while un-sober, if you like.

      source

      • morrowind@lemmy.ml ⁨1⁩ ⁨year⁩ ago

        I… Don’t think I’ve heard anyone claim authority shouldn’t exist.

        source

        • -> View More Comments
  • IrateAnteater@sh.itjust.works ⁨1⁩ ⁨year⁩ ago

    That’s not really relevant here. This is more of a “genie is out of the bottle and now we have to learn how to deal with it situation”. The idea and technology of bots and AI training already exists. There’s no socioeconomic system that is going to magically make that go away.

    source

    • revv@lemmy.blahaj.zone ⁨1⁩ ⁨year⁩ ago

      I think the point you’re missing is that without the monetary incentive that arises under capitalism, there would be very little drive for anyone to build these wasteful AI systems. It’s difficult to imagine a group of people voluntarily amassing and then using the resources necessary for “AI” absent the desire to cash in on their investment. So you’re correct that an alternative economic system won’t “magically” make LLMs go away. I think it unlikely, however, that such wasteful nonsense would be used on any meaningful scale absent the perverse incentives of capitalism.

      source

      • Buelldozer@lemmy.today ⁨1⁩ ⁨year⁩ ago

        It’s difficult to imagine a group of people voluntarily amassing and then using the resources necessary for “AI” absent the desire to cash in on their investment.

        No imagination necessary.

        I mean Dmitry Pospelov was arguing for AI control in the Soviet Union clear back in the 70s.

        source

        • -> View More Comments
    • melpomenesclevage@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

      I don’t need it to not exist. I need it to stay the fuck out of everyone’s lives unless they work in a lab of some kind.

      source

      • desktop_user@lemmy.blahaj.zone ⁨1⁩ ⁨year⁩ ago

        everyone remembers tomogatchi, they were like a digital houseplant.

        source

        • -> View More Comments
  • doodledup@lemmy.world ⁨1⁩ ⁨year⁩ ago

    Lost on Lemmy?

    source

    • limer@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

      He is not wrong. Unless people start to take steps , the dependency of tech will be used to chain most of us. Granted, these chains will be the kindest and gentlest chains seen in a long time.

      Social revolution lives on in decentralized services, like this; the true battles will be later though. This year is a mild warm up. I can’t imagine the challenges that await many

      source
    • melpomenesclevage@lemmy.dbzer0.com ⁨1⁩ ⁨year⁩ ago

      no, responding to a post about exactly that thing.

      source