i configured asn blocking on my website and my files website (not yet for all my services)
hopefully those pesky crawlers will be gooone