Comment

Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.

<- View Parent

sudo@programming.dev ⁨1⁩ ⁨day⁩ ago

This is what I’ve kept saying about POW being a shit bot management tactic. Its a flat tax across all users, real or fake. The fake users are getting making money to access your site and will just eat the added expense. You can raise the tax to cost more than what your data is worth to them, but that also affects your real users. Nothing about Anubis even attempts to differentiate between bots and real users.

If the bots take the time, they can set up a pipeline to solve Anubis tokens outside of the browser more efficiently than real users.

source

Sort:hotnew top

black_flag@lemmy.dbzer0.com ⁨1⁩ ⁨day⁩ ago
Yeah but ai companies are losing money so in the long run Anubis seems like it should eventually return to working.

source
- sudo@programming.dev ⁨1⁩ ⁨day⁩ ago
  Costs of solving PoW for Anubis is absolutely not a factor in any AI companies budget. Just the costs of answering one question is millions of times more expensive than running sha256sum for Anubis.
  
  Just in case you’re being glib and mean the businesses will go under regardless of Anubis: most of these are coming from China. China is absolutely will keep running these companies at a loss for the sake of strategic development.
  
  source
  - black_flag@sh.itjust.works ⁨1⁩ ⁨day⁩ ago
    Thanks for the info 👍 would not have thought Anubis would be so irrelevant
    
    source
OpenPassageways@lemmy.zip ⁨1⁩ ⁨day⁩ ago
What the alternative?

source
- sudo@programming.dev ⁨1⁩ ⁨day⁩ ago
  Not much for open source solutions. A simple captcha however would cost scrapers more to crack than Anubis.
  
  But when it comes to “real” bot management solutions: The least invasive solutions will try to match User-Agent and other headers against the TLS fingerprint and block if they don’t match. More invasive solutions will fingerprint your browser and even your GPU, then either block you or issue you a tracking cookie which is often pinned to your IP and user-agent. Both of those solutions require a large base of data to know what real and fake traffic actually looks like. Only large hosting providers like CloudFlare and Akamai have that data and can provide those sorts of solutions.
  
  source