Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 1 year agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 1 year agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
catloaf@lemm.ee 1 year ago
That’s what ddos protection is for.