The local LLM is run on the homelab, just like immich is run on your homelab and doesnt talk to google photos is any way, its the same for my model, self contained, inhouse with no data leaving my network
Comment on How my AI Agent views and maintains "our" homelab
crash_thepose@lemmy.ml 1 day ago
When you have a local llm, is it still relying on the energy resources of open ai or the like? Sorry for the dumb question
variety4me@lemmy.zip 1 day ago
crash_thepose@lemmy.ml 1 day ago
Meaning you download the entire large language model?
variety4me@lemmy.zip 1 day ago
Yes, the download link for the model is in the original post
crash_thepose@lemmy.ml 1 day ago
So it does use the resources of an LLM company like Google or open ai?
atzanteol@sh.itjust.works 21 hours ago
Does… It matter?
Energy use is energy use no?
The energy required to do inference (i.e. amount it questions and the like) is no worse than doing some gaming for a short period of time.
That said it’s probably less efficient to run locally since anthropic and openai have been getting more efficient data center hardware from nvidia compared to consumer desktop gpus.
crash_thepose@lemmy.ml 17 hours ago
Well, questions of efficiency seem like it does matter, no?
harmbugler@piefed.social 19 hours ago
Consider also the energy and water they use to cool the datacentre equipment, which doesn’t usually happen at home.
Originally training the model had used the energy resources of that original corporation or whatever. But when you download that model and start running it on your own hardware, you are using your own energy.
Think of it kind of like some software like Jellyfin. When the developers write the software, they do so using their own electricity. But when you download Jellyfin and actually run the software on your own hardware, you are now only using your electricity, not the developer’s electricity at all.
crash_thepose@lemmy.ml 17 hours ago
Thank you for explaining!