Comment on Have clankers visited my blog one hundred twenty-one sexagintillion eight hundred ten novemquinquagintillion times so far in November??
carrylex@lemmy.world 4 weeks ago
Can you just turn the robots.txt into a click wrap agreement to charge robots high fees for access above a certain threshold?
why do a agreement when you can serve a zip bomb :D
Puts the full EU regulations in robot.txt
Cort@lemmy.world 4 weeks ago
Can you just turn the robots.txt into a click wrap agreement to charge robots high fees for access above a certain threshold?
carrylex@lemmy.world 4 weeks ago
why do a agreement when you can serve a zip bomb :D
Valmond@lemmy.world 4 weeks ago
Puts the full EU regulations in robot.txt