The local LLM is run on the homelab, just like immich is run on your homelab and doesnt talk to google photos is any way, its the same for my model, self contained, inhouse with no data leaving my network
Comment on How my AI Agent views and maintains "our" homelab
crash_thepose@lemmy.ml 3 weeks ago
When you have a local llm, is it still relying on the energy resources of open ai or the like? Sorry for the dumb question
variety4me@lemmy.zip 3 weeks ago
crash_thepose@lemmy.ml 3 weeks ago
Meaning you download the entire large language model?
variety4me@lemmy.zip 3 weeks ago
Yes, the download link for the model is in the original post
crash_thepose@lemmy.ml 3 weeks ago
So it does use the resources of an LLM company like Google or open ai?
atzanteol@sh.itjust.works 3 weeks ago
Does… It matter?
Energy use is energy use no?
The energy required to do inference (i.e. amount it questions and the like) is no worse than doing some gaming for a short period of time.
That said it’s probably less efficient to run locally since anthropic and openai have been getting more efficient data center hardware from nvidia compared to consumer desktop gpus.
crash_thepose@lemmy.ml 3 weeks ago
Well, questions of efficiency seem like it does matter, no?
harmbugler@piefed.social 3 weeks ago
Consider also the energy and water they use to cool the datacentre equipment, which doesn’t usually happen at home.
Originally training the model had used the energy resources of that original corporation or whatever. But when you download that model and start running it on your own hardware, you are using your own energy.
Think of it kind of like some software like Jellyfin. When the developers write the software, they do so using their own electricity. But when you download Jellyfin and actually run the software on your own hardware, you are now only using your electricity, not the developer’s electricity at all.
crash_thepose@lemmy.ml 3 weeks ago
Thank you for explaining!