Well, that depends on workload and the employer. If you are one of the lucky ones where it’s just important that shit gets done on time, it would result it lower usage. That’s on the employer, not on the LLM.
3W/request (4W if you include training the model) is nothing compared to what we use in our everyday life, and it’s even less when looking what other activities consume. Noone would have an issue with you running a blender for 30 seconds, even tho it’s the same energy usage as an Chatbot request.
HubertManne@piefed.social 5 days ago
see again you compare something someone might do once a day and most people do hardly ever. Using a blender. With something used many times constantly through out the day. Web searching. Even before ai datacenters were a massive use of energy. Now im not the type to say throw it all away but I will be careful in my usage till im sure its worth it. this is going to require the vendors to put out data on energy usage. Its new enough that im sure more and better chips will be able to reduce the energy it takes. You have to realize your talking to someone who walks if I can, then bikes as a second option, and finally takes public transit. I avoid driving and planes unless I absolutely have to. Im in tech so I will be using it but it will likely follow the same curve as previous technology has but maybe not given smartphones and apps would be the most recent things before ai and I use those only if I absolutely have to.
a_wild_mimic_appears@lemmy.dbzer0.com 5 days ago
i had no car my entire life and have flown 4 times (i like trains lol).
Ok, something nearly everyone does - washing clothes.
A 500W washing machine uses this amount of energy in 10.8 seconds during the spin cycle.
Using a dryer? 4.3 seconds@3000W.
If we look at the equivalent mechanical energy, lets look at your bike (you gotta eat to offset the energy loss, causing emissions through agriculture and cooking!)
You can generate a Chatbot response by pedaling ~2 minutes@100W power!
The chips are already pretty optimized, since they are in principle not different to high-end gaming GPU’s. I have a 3070 TI, and i can generate a complex response locally in under a minute @ 290W TDP, 30-40 seconds if i use something like qwen 2.5, which aligns pretty well with what i’ve said before. Playing a video game uses a lot more power.
HubertManne@piefed.social 5 days ago
2 minutes for every query and im 100% not using it and I think these numbers like 2 min of biking aren't really based on anything. It will help again if the industry makes it a point to track and publish energy usage.