Hi there, I’m thinking about getting into self-hosting. I already have a Jellyfin server set up at home but nothing beyond that really. If you have a few minutes, how can self-hosting help in the context of OPs post? Do you mean hosting LLMs on Ollama?
Comment on Chatgpt shared link searchable
FauxLiving@lemmy.world 3 weeks agoIf you don’t want corporations to use you chats as data, don’t use corporate hosted language models.
Even non-public chats are archived by OpenAI, and the terms of service of ChatGPT essentially give OpenAI the right to use your conversations in any way that they choose.
You can bet they’ll eventually find ways to monetize your data at some point in the future. If you think GoogleAds is powerful, wait until people’s assistants are trained with every manipulative technique we’ve ever invented and are trying to sell you breakfast cereals or boner pills…
You can’t uncheck that box except by not using it in the first place. But people will sell their soul to a company in order to not have to learn a little bit about self-hosting
puck@lemmy.world 3 weeks ago
BreadstickNinja@lemmy.world 3 weeks ago
Yes, Ollama or a range of other backends (Ooba, Kobold, etc.) can run LLMs locally. Huggingface has a huge number of models suited to different tasks like coding, storywriting, general purpose, and so on. If you run both the backend and frontend locally, then no one monetizes your data.
The part I’d argue that the previous poster is glazing over a little bit is performance. Unless you have an enterprise-grade GPU cluster sitting in your basement, you’re going to make compromises on speed and/or quality relative to the giant models that run on commercial services.
tal@lemmy.today 3 weeks ago
It’s also going to cost more, because you almost certainly are only going to be using your hardware a tiny fraction of the time.
BreadstickNinja@lemmy.world 3 weeks ago
Possibly, yes. There are models that will run on consumer-grade GPUs that you might already have or might have purchased anyway, where you might say there’s no incremental cost. But the issue is that the performance will be limited. The models are forgetful and prone to getting stuck in loops of repeated phrases.
So if instead you custom-build a workstation with two 5090s or a Pro 6000 or something that pushes you up to the 100 GB VRAM tier, then absolutely, just as you said you’ll be spending thousands of dollars that probably won’t pay back relative to renting cloud GPU time.
puck@lemmy.world 3 weeks ago
Thanks for the info. Yeah, I was wondering what kind of hardware you’d need to host LLMs locally with decent performance and your post clarifies that. I doubt many people would have the kind of hardware required.
Electricd@lemmybefree.net 3 weeks ago
This is basically a “if you don’t want your data to be used, run your own internet” comment
It’s just not doable for pretty much everyone
Allero@lemmy.today 3 weeks ago
Modern LLMs can serve you for most tasks while running locally on your machine.
Something like GPT4ALL will do the trick on any platform of your choosing if you have at least 8gb of RAM (and for most people nowadays it’s true).
It has a simple, idiot-proof GUI and doesn’t collect data if you don’t allow it to. It’s also open source.
Electricd@lemmybefree.net 3 weeks ago
If you want actual good features like deep research or chain of thought, eh, not sure it’s a good choice
The models will also not be very powerful
null@lemmy.nullspace.lol 3 weeks ago
And you don’t need any of that. You don’t even need a local LLM.
So if you decide you want it, then that’s on you, and you have made the choice to give up your data.