Want a more private ChatGPT alternative that runs offline? Check out Jan

Comments

• PerogiBoi@lemmy.ca ⁨9⁩ ⁨months⁩ ago

  Also check out LLM Studio and GPT4all. Both of these let you run private ChatGPT alternatives from Hugging Face and run them off your ram and processor (can also offload to GPU).

  source

  • Just_Pizza_Crust@lemmy.world ⁨9⁩ ⁨months⁩ ago

    I’d also recommend Oobabooga if you’re already familiar with Automatic1111 for Stable diffusion. I have found being able to write the first part of the bots response gets much better results and seems to make up false info much less.

    source

    • FaceDeer@kbin.social ⁨9⁩ ⁨months⁩ ago

      There's also koboldcpp, which is fairly newbie friendly.

      source

      • -> View More Comments
  • tubbadu@lemmy.kde.social ⁨9⁩ ⁨months⁩ ago

    Are they as good as chatgpt?

    source

    • PerogiBoi@lemmy.ca ⁨9⁩ ⁨months⁩ ago

      Mistral is thought to be almost as good. I’ve used the latest version of mistral and found it more or less identical in quality of output.

      It’s not as fast though as I am running it off of 16gb of ram and an old GTX 1060 card.

      source

      • -> View More Comments
    • tsonfeir@lemm.ee ⁨9⁩ ⁨months⁩ ago

      No.

      source
• stevedidWHAT@lemmy.world ⁨9⁩ ⁨months⁩ ago

  Open source good, together monkey strong 💪🏻

  Build cool village with other frens, make new things, celebrate as village

  source

  • Tja@programming.dev ⁨9⁩ ⁨months⁩ ago

    Apes together *

    source
• TootSweet@lemmy.world ⁨9⁩ ⁨months⁩ ago

  It seems like usually when an LLM is called “Open Source”, it’s not. It’s refreshing to see that Jan actually is, at least.

  source
• drislands@lemmy.world ⁨9⁩ ⁨months⁩ ago

  Sure, Jan.

  source
• wetferret@lemmy.world ⁨9⁩ ⁨months⁩ ago

  I would also reccommend faraday.dev as a way to try out different models locally using either CPU or GPU. I believe they have a build for every desktop OS.

  source
• randon31415@lemmy.world ⁨9⁩ ⁨months⁩ ago

  I have recently been playing with llamafiles, particularly Llava which, as far as I know, is the first multimodal open source llm (others might exist, this is just the first one I have seen). I was having it look at pictures of prospective houses I want to buy and asking it if it sees anything wrong with the house.

  The only problem I ran into is that window 10 cmd doesn’t like the sed command, and I don’t know of an alternative.

  source

  • ramjambamalam@lemmy.ca ⁨9⁩ ⁨months⁩ ago

    Would it help to run it under WSL?

    source
  • halva@discuss.tchncs.de ⁨9⁩ ⁨months⁩ ago

    might be a good idea to use windows terminal or cmder and wsl instead of windows shells

    source
  • Tja@programming.dev ⁨9⁩ ⁨months⁩ ago

    WSL2

    source
  • altima_neo@lemmy.zip ⁨9⁩ ⁨months⁩ ago

    Powershell, maybe?

    source
• tubbadu@lemmy.kde.social ⁨9⁩ ⁨months⁩ ago

  Are they as good as chatgpt?

  source

  • Infiltrated_ad8271@kbin.social ⁨9⁩ ⁨months⁩ ago

    The question is quickly answered as none is currently that good, open or not.

    Anyway it seems that this is just a manager. I see some competitors available that I have heard good things about, like mistral.

    source

    • Bipta@kbin.social ⁨9⁩ ⁨months⁩ ago

      Local LLMs can beat GPT 3.5 now.

      source

      • -> View More Comments