SolidShake@lemmy.world 3 days ago
I don’t think regular people really understand the power needed for AI. It’s often taught that we just have it. But not where it comes from.
People keep telling us that ai energy use is very low, but at the same time, companies keep building more and more giant power hungry datacenters. Something simply doesn’t add up.
Sure, a small local model can generate text at low power usage, but how useful will that text be, and how many people will actually use it? What I see is people constantly moving to the newest, greatest model, and using it for more and more things, processing more and more tokens. Always more and more.
Each datacenter is set to handle millions of users, so it concentrates all the little requests into very few physical locations.
The tech industry further amplifies things with ambient LLM invocation. You do a random google search, it implicitly does an LLM unasked. When a user is using an LLM enabled code editor, it’s making LLM requests every few seconds of typing to drive the autocomplete suggestions. Often it has to submit a new LLM request before the old one even completed because the user typed more while the LLM was chewing on the previous input.
So each LLM invocation may be reasonable, but they are being concentrated impact wise into very few places and invocations are amplified by tech industry being overly aggressive about overuse for the sake of 'ambient magic.
I don’t think regular people really understand how little 3W per request is. It’s the energy you take up by eating 3kcal. Or what your WiFi router uses in half an hour. Or your clothes dryer in 5 seconds.
I wonder how the power usage of running an LLM locally compares to playing a modern game at high settings. Can they be very different?
I would compare LLM to bit mining. But I’m not an expert
slazer2au@lemmy.world 3 days ago
True, but most people don’t realise how little not printing an email ‘helped’ the environment.
SolidShake@lemmy.world 3 days ago
It would have been significant if a lot of people did it.
IndiBrony@lemmy.world 3 days ago
I’m doing my part. Can’t remember the last time I had to print anything.
Trainguyrom@reddthat.com 2 days ago
My printer died so I go to the library to print when I need to. For ¢25 per color print or ¢10 per black and white print it’s a lot cheaper than buying another printer