Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 8 months agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 8 months agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
catloaf@lemm.ee 8 months ago
That’s what ddos protection is for.