Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 1 month agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 1 month agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
catloaf@lemm.ee 1 month ago
That’s what ddos protection is for.