Local runs on device, so no need to connect to a big data center that chugs lots of water and all those other problems. Of course, because it’s a smaller far tinier model it’s nowhere near as accurate, but especially for things like this you don’t really need a big accurate LLM model.
I think I also though I should warrant a disclaimer that I am a Software Developer, not a AI Developer. So there’s far less backing then from my perspective than someone who works with this stuff for a living
MirrorGiraffe@piefed.social 2 days ago
I’m also a sw engineer so we’re both guessing 😅
I’m guessing those dates centers use that water for cooling whereas most home computers run an electric fan. And furthermore they probably use less electricity per token as they want to maximize profits. I don’t have any numbers to back my hunch up but I’m pretty sure the environment would suffer more if everyone is running their own.
I probably missed a lot of factors such as what type of energy the centers run contra what average Joe runs etc.