sorry. there were two conversations and Im getting confused. Are you talking local where I don't have the overhead? Or using them online where I am worried about the energy usage?
How much further down than 3W/request can you go? i hope you don’t let your microwave run 10 seconds longer than optimal, because that’s exactly the amount of energy we are talking about. Or running a 5W nightlight for a bit over half an hour.
LLM and Image generation are not what kills the climate. What does are flights, cars, meat, and bad insulation of houses leading to high energy usage in winter. Even if we turned off all GenAI, it wouldn’t even leave a dent compared to those behemoths.
HubertManne@piefed.social 5 days ago
a_wild_mimic_appears@lemmy.dbzer0.com 5 days ago
that’s online, what is used in the data centers per request. local is probably not so different, depending on device - different devices have different architectures which might be more or less optimal, but the cooling is passive.
HubertManne@piefed.social 5 days ago
Yeah the thing is its not comparing each request to an airline flight its comparing each one to a web search. Its utility is not that much greater its just a convenience. Its like with bitcoin where its about energy per transaction compared to a credit card transactions. I mean I search the web everyday a whole bunch and way more when im working.
a_wild_mimic_appears@lemmy.dbzer0.com 5 days ago
Whether its useful or not is another discussion, but if you used a LLM to write an email in 2 minutes that you would use 10 minutes for (including searches and whatever), you actually generate LESS CO2 than the manual process:
- PC, 200W/h, 10 min: 33W
- Monitor, 30W/h: 5W
- Google searches, lets say 3, about 0,3W/Search: 1W
equals ~40W
compared to:
- PC, 200W/h, 2 min: ~7W
- Monitor, 30W/h: 1W
- ChatGPT, 3 requests (normally you would expect less requests than google searches, but for the argument…): 9W
equals ~17W.
And that is excluding many other factors like general energy costs for infrastructure, which tilt the calculation further in chatgpts favor.
EVERYTHING we do creates emissions one way or another, we create emissions by simply existing too; it’s important to set things into perspective. Both upper examples compare to running a 1000W microwave for either 2:20 min or 1:05 min. You wouldn’t be shocked by those values.
jj4211@lemmy.world 5 days ago
A local LLM probably costs about as much as the online LLM, but that usage is so widely distributed it’s not as impactful to any specific locations.
fading_person@lemmy.zip 5 days ago
Where is this 3W from? W isn’t even an energy unit, but a power unit.
a_wild_mimic_appears@lemmy.dbzer0.com 5 days ago
sorry, it should be 3 Wh, you are correct of course. The 3Wh come from here:
Every source here stays below 3Wh, so it’s reasonable to use 3Wh as an upper bound. (I wouldn’t trust Altmans 0,3 Wh tho lol)
fading_person@lemmy.zip 5 days ago
Thanks for linking the sources. I will take a look into that