Comment on Codeberg: army of AI crawlers are extremely slowing us; AI crawlers learned how to solve the Anubis challenges.
Crawlers aren’t LLMs; they can do arbitrary computations (whatever the target demands to access resources).