Comment on FediDB has stoped crawling until they get robots.txt support

<- View Parent
Randelung@lemmy.world ⁨2⁩ ⁨months⁩ ago

We can’t afford to wait at every sop, yeld, or one vay sign!

source
Sort:hotnewtop