What if I made a static site using Github pages hosting? Will having a robots.txt in my root folder ward off Google bhoots (devils)?
Comment on Google is no longer asking — feed the AI or you’re not in search results
ColinHayhurst@lemmy.world 4 weeks agoPut should put these entries into your robots.txt file.
To block the Google search crawler use for all of your site:
User-agent: Googlebot Disallow: /
To block the Google AI crawler use:
User-agent: Google-Advanced Disallow: /
Snowcano@startrek.website 4 weeks ago
You rock, thank you!