Smokey's Simple Guide To Search Engine Alternatives

Submitted ⁨⁨1⁩ ⁨year⁩ ago⁩ by ⁨Smokeydope@lemmy.world⁩ to ⁨technology@lemmy.world⁩

Smokey’s Simple Guide To Search Engine Alternatives

This post was inspired by the surge in people mentioning the new Kagi Search engine on various lemmy comments. I happen to be somewhat knowledgeable on the topic and wanted to tell everyone about some other alternative search engines available to them, as well as the difference between meta-search engines and true search engines.

Understanding Search Engines Vs. Meta-Search Engines

There are many alternative search engines floating around that people use, however most of them are meta search engines. Meaning that they are a kind of search result reseller, middle men to true search engines. They query the big engines for you and aggregate their results.

Examples of Meta-search engines:

Duckduckgo: bing, Ecosia: Bing + Google Kagi(kagi.com): Google, Mojeek, Yandex, Marginalia, 10$/month premium for unlimited searches & email signup SearXNG:Too many to list, basically all of them, configurable, Free & Open Source Software AGPL-3.0 Startpage: Google + Bing Swisscows: Bing Qwant:bing

##True Search Engines & The Realities Of Web-Crawling As you can see, the vast majority of alternative search engines rely on some combination of Google and Bing. The reason for this is that the technology which powers search engines, webcrawling and indexing, are extremely computationally heavy and non-trivial things. They require powerful enterprise machines and the more popular the service (as in the more people connecting to and using it per second) the more internet bandwith and processing power is needed. It takes a lot of money to pay for power, maintenance, and development/security.

Examples of True Search Engines:

Bing: Owned by Microsoft Google: Owned by Google/Alphabet Mojeek Owned by Mojeek LLC Yandex: Owned by Yandex Inc YaCy: Free & Open Source Software [GPL-2.0]www.gnu.org/licenses/…/gpl-2.0.en.html), powered by peer to peer technology, created by Michael Christen, Marginalia Search: Free & Open Source Software AGPL-3.0, developed by Marginalia/ Martin Rue

This is a big financial ask for most companies interested in making a profit out of the gate, they determine its worth just paying Google and Bing for access to their enormous pre-existing infrastracture without the headaches of dealing with maintenance and security risk.

#About Some Of The Free & Open Source Search Engines SearX/SearXNG is a free and open source, highly customizable, and self-hostable meta search engine. SearX instances act as a middle man, they query other search engines for you, stripping all their spyware ad crap and never having your connection touch their servers. ##Decentralization Federation and Instances As Lemmy users, you should at least vaugely understand the power of a decentralized service spread out among many individually operated/maintained instances that can cooperate with eachother.

Companies that live and die by profit margins have to concern themselves with the choice of owning their own massive computer infastractures or paying for lots of access to someone elses. Decentralized free and open source search engines that can be self-hosted by thousands of computer hobbiest from around the world can spread collective user load and eat the meager maintenance cost of hosting an instance on a old desktop computer or extremely energy efficent Single Board Computer.

Additionally the spread of users across multiple instances helps prevent any one of them from exceeding the free/cheap allotment of api calls in the case of meta-search engines like SearXNG or being rate limited like 3rd party youtube scrapers such as Invidious and Piped.

In the case of YaCy decentralization is also federated, all individual yacy instances communicate with eachother through p2p to act as one big collective web crawler and indexer making it the only true independent FOSS search engine I know of.

##Self-Hosting For Maximum Privacy Of course you have to trust the SearX instance provider with your information. Trust is a big concern with every engine you use, because while they can promise to not log anything or sell your info for profit, they often provide no way of proving those claims to be true beyond ‘just trust me bro’. If you are absolutely concerned with privacy and knowledgeable with computers then self hosting FOSS software from your own instance is the best option to maintain control of your data.

Free As In Freedom, People vs Company Run Services

I personally trust some foss loving sysadmin that host social services for free out of alturism, who also accepts hosting donations, whos server is located on the other side of the planet, with my query info over Google/Alphabet any day. I have had several communications with Marginalia over several years now through the gemini protocol and small web, they are more than happy to talk over email. have a human conversation with your search engine provider thats just a knowlegable every day joe who genuinely believes in the project and freely dedicates their resources to it. Consider sending some cash their way to help with upkeep if you like the services they provide.

Heres a list of all public searx instances, I personally prefer to use paulgo.io All SearX instances are configured different to index different engines. If one doesn’t seem to give good results try a few others.

Did I mention it has bangs like duckduckgo? If you really need google like for maps and buisness info just use !!g in the query

[search.marginalia.nu](Marginalia Search) is a completely novel search engine written and hosted by one dude that aims to prioritize indexing lighter websites little to no javascript as these tend to be personal websites and homepages that have poor SEO and the big search engines won’t index well. If you remember the internet of the early 2000s and want a nostalgia trip this ones for you. Its also open source and self-hostable

Finally, YaCy is another completely novel search engine that uses peer-to-peer technology to power a big webcrawler which prioritizes indexes based off user queries and feedback. Everyone can download yacy and devote a bit of their computing power to both run their own local instance and help out a collective search engine. Companies can also download yacy and use it to index their private intranets.

They have a public instance available through a web portal. To be upfront, YaCy is not a great search engine for what most people usually want, which is quick and relevant information within the first few clicks. But, it is an interesting use of technology and what a true honest-to-god community-operated search engine looks like untainted by SEO scores or corporate money-making shenanigans.

I hope this has been informative to those who believe theres only a few options to pick from, I know these options are so unknown to most people.

source

Comments

Sort:hotnew top

Xavier@lemmy.ca ⁨1⁩ ⁨year⁩ ago
Excellent writeup! With constant updates to boot 🥳

I’m saving it for future reference.

Thank you for putting your time and effort on this.

source
heygooberman@lemmy.today ⁨1⁩ ⁨year⁩ ago
This is awesome! I recently switched over to Kagi because of several Lemmy posts mentioning it as an alternative. However, I really appreciate you sharing your preferred SearXNG instance. I have it added to my browser’s list of search engines and may switch to it from time-to-time to see how different it is from Kagi. If there really isn’t much of a difference, then I may switch to it after my Kagi membership ends.

source
TheDuckPrince@lemm.ee ⁨4⁩ ⁨months⁩ ago
I discovered searx via this post, currently I use startpage which is the only one that restiuses me results as similar as possible to google. Is it possible to get a similar result with Seraxng? because it seems to me that the results are always very strange, it returns sites that are often inconsistent or unfamiliar. To give an example, if I search for searx in google the first result is wikipedia and the fourth searx.space, while in searxng the results are different depending on the instance used. Thank you

source
- Smokeydope@lemmy.world ⁨4⁩ ⁨months⁩ ago
  Yes it is absolutely possible to get similar results with searxng to start page, in fact some instances allow you to aggregate startpage results directly.
  
  I found priv.au as an example of an instance that can directly aggregate startpage.
  
  So the trick to the instance weirdness going on is that each instance has its own set of default engines set to aggregate from. For example one searxng instance may want to only aggregate google and bing, while another may want to aggregate only independent search engines that don’t use google or bing such as YaCy and Qwant.
  
  Here’s what you can do about that: The secret to overcoming this and dialing in the search results you want is to realize you can actually configure each searxng instance to aggregate the engines you want while disabling the default ones you don’t. All searxng/searxng instances have a preferences menu usually a gear icon in the top right corner. Or you can go to searxng-example.com/preferences . Once in preferences go to the ‘engines’ section from there you can tick the engines you want to use.
  
  Some instances save your settings as cookies, some instances save your settings as a sub URL for that instance. The priv.au instance I mentioned saves your settings as cookies.
  
  Hope this helps.
  
  source
Mubelotix@jlai.lu ⁨1⁩ ⁨year⁩ ago
Let me mention my own, a peer-to-peer (not federated) search engine for the decentralized web (ipfs)

source
- Smokeydope@lemmy.world ⁨1⁩ ⁨year⁩ ago
  Very nice, I may do a post about alternative protocols and the small web, and don’t mind linking to your engine in the IPFS section,
  
  source
  - Mubelotix@jlai.lu ⁨1⁩ ⁨year⁩ ago
    I would be very pleased, thanks
    
    source
alphacyberranger@lemmy.world ⁨1⁩ ⁨year⁩ ago
Good post

source
doctortofu@reddthat.com ⁨1⁩ ⁨year⁩ ago
Great list! Is there a way to easily add SearXNG as a default search engine in Firefox for Android? That’s the only thing I’d like to make it even more convenient

source
- Viking_Hippie@lemmy.world ⁨1⁩ ⁨year⁩ ago
  I was wondering the same thing but didn’t really understand from your comment how it can be done… Could you ELI5 or link me to a guide?
  
  source
  - doctortofu@reddthat.com ⁨1⁩ ⁨year⁩ ago
    What I did was first set all the options up and save them, and then go to “Cookies” in the search engine settings and copy the “Search URL of the currently saved preferences”. Then, open Firefox settings -> Search -> Manage alternative search engines -> Add search engine and paste the thing into the “Search string URL” field, type in the name and press save. After that I just set it as default, and it works.
    
    Hope this helps!
    
    source
    -> View More Comments
Mojeek@lemmy.ml ⁨10⁩ ⁨months⁩ ago
good write up, we’re Mojeek Limited, not Mojeek LLC; LLC as a formation in the UK is called a “Private Limited Company” (PLC) which is contracted to .ltd or limited.

source
- Smokeydope@lemmy.world ⁨10⁩ ⁨months⁩ ago
  Thank you! Apologies for the mistake, it has now been corrected to .ltd.
  
  source
  - Mojeek@lemmy.ml ⁨9⁩ ⁨months⁩ ago
    ah no bother at all, not everyone is gonna be across every single kind of company and they are functionally very similar!
    
    source
Imhotep@lemmy.world ⁨1⁩ ⁨year⁩ ago
Qwant has web crawlers. It started with indexing the German and French web, and the plan was to progressively rely less on Bing

Since the search engine is a failure I suppose they didn’t develop it further.

source
- Smokeydope@lemmy.world ⁨1⁩ ⁨year⁩ ago
  Qwant is a weird one for sure, from what I understand most of its life it relied on bing and only started making efforts to build up its own crawlers and such after 2019. Making it a case of a meta search engine trying to become more of a true search engine. They still use bing even if minimally. I can maybe put in a small footnote about it but I’m not sure if the average person would really care about qwants insider baseball.
  
  source
  - Imhotep@lemmy.world ⁨1⁩ ⁨year⁩ ago
    I’m having a hard time figuring it out also. An article says in 2018 they had 20B pages indexed… but they didn’t use them?
    
    People won’t understand the difference between meta search engine and real one most likely. But if they did, i believe some would choose the “60% independent”, the same way they choose a “60% recycled material”
    
    source
blackfire@lemmy.world ⁨1⁩ ⁨year⁩ ago
This is a good list although i generally wouldnt bother listing yacy. Its only as good as the people adding to the list and thats not a lot.

source
- ElectroVagrant@lemmy.world ⁨1⁩ ⁨year⁩ ago
  
  This is a good list although i generally wouldnt bother listing yacy. Its only as good as the people adding to the list and thats not a lot.
  
  Isn’t that last point a good reason to mention it, to possibly increase the amount of people contributing?
  
  source
  - blackfire@lemmy.world ⁨1⁩ ⁨year⁩ ago
    It’s been around for a very long time and not gotten any traction. I just don’t think it can gather enough of a community to make it the goto source.
    
    source
sic_semper_tyrannis@feddit.ch ⁨1⁩ ⁨year⁩ ago
Great post! Now I have a better understanding of the subject. Another meta search engine I found is 4get.ca

source
steeznson@lemmy.world ⁨11⁩ ⁨months⁩ ago
Dropping into this thread late to mention how much I have enjoyed using marginalia.nu - the whole experience is just a joy on a desktop. Not strictly related to search but their wikipedia-type encyclopedia has has been improving a lot recently too.

source
BlackSkinnedJew@lemmynsfw.com ⁨11⁩ ⁨months⁩ ago
I’m using presearch.com and everything goes smooth.

source
Blisterexe@lemmy.zip ⁨1⁩ ⁨year⁩ ago
What about brave search?

source