Probably just a misconfigured bot and nothing special.
Google Using Generic & Undocumented Google Crawler
Submitted 7 months ago by tonytins@pawb.social to technology@lemmy.world
https://www.seroundtable.com/google-undocumented-google-crawler-39915.html
Comments
Toes@ani.social 7 months ago
LodeMike@lemmy.today 7 months ago
The crawler user agent is just named “Google,”
skeezix@lemmy.world 7 months ago
Very sneaky.
Flyswat@lemmy.dbzer0.com 7 months ago
Can we get rid of the current model where websites are financed by ads and have instead these AI companies pay the websites they crawl?
I could really do with less ads if they are going to crawl the web anyway.