‘s/ly (.) (.)/\2/’
There is nothing good about Google results. They haven’t been usable for years.
Comment on Why I recommend against Brave.
yqccv9v3li8o@lemmy.sdf.org 3 weeks ago
I agree that Brave is problematic and most of the criticisms are also legitimate. But the point about the crawlers is not really a problem. If you respect the wish of websites to be indexed exclusively by Google, you support Google’s monopoly and prevent alternative search engines from having similarly good results.
FauxPseudo@lemmy.world 3 weeks ago
Jerry@feddit.online 3 weeks ago
I see no legitimate reason for not using a User Agent string, like all the other crawlers use, other than the desire to hide the crawler and make it difficult to block.
I don't accept his explanation. I see it as gaslighting.
yqccv9v3li8o@lemmy.sdf.org 3 weeks ago
Why should the crawler be blockable? That only brings disadvantages for a search engine.
unautrenom@jlai.lu 3 weeks ago
It’s not about ‘Google’ vs ‘the other sear h engines’. It’s about transparency. You’ve probably read some news about how AI crawlers have been destroying infrastrucure and half the time does NOT declare themselves as crawlers in their UA.
Can confirm that nealy 90% (read hundred of thousands) of daily visits to several of my websites are made by crawlers from datacenters and I HATE not knowing whose who. Because when I don’t know, I block and report. Website owners already have enough between AI, Page Rankings, and Research Agencies who all exploit free infra for theie own business.
Do I make exception for Search Engine crawlers? Yeah, I do. I’ve seen Google, Bing, and Mojeek, but weirdly enough, never Brave. Now I know why. And frankly, if they can’t be bothered to be transparent about their crawlings, then I won’t be bothered to make exceptions for them. They’re freeloading just as much as the rest. If they act like shady chinese crawlers, then they have no right to go pikachu face when they’re treated like one.
Jerry@feddit.online 3 weeks ago
Well said
Ulrich@feddit.org 3 weeks ago
Brave doesn’t have AI crawlers, they have search index crawlers.
Ulrich@feddit.org 3 weeks ago
They explained the reason in the comment you just replied to.