Comment on FediDB has stoped crawling until they get robots.txt support
rimu@piefed.social 3 weeks ago
lol FediDB isn't a crawler, though. It makes API calls.
Comment on FediDB has stoped crawling until they get robots.txt support
rimu@piefed.social 3 weeks ago
lol FediDB isn't a crawler, though. It makes API calls.