Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 3 months agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
Comment on AI crawlers cause Wikimedia Commons bandwidth demands to surge 50%.
grysbok@lemmy.sdf.org 3 months agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
catloaf@lemm.ee 3 months ago
That’s what ddos protection is for.