Comment on How Much Energy Does AI Use? The People Who Know Aren’t Saying
Xaphanos@lemmy.world 6 months ago
A problem is that the information is not in the hands of the company selling the AI. The actual hardware is often owned by service providers and independent data centers.
silence7@slrpnk.net 6 months ago
They know exactly what the power consumption of that hardware is though. This isnt tough to figure out just because you use a cloud provider
Xaphanos@lemmy.world 6 months ago
Zeoic@lemmy.world 6 months ago
Im sure they can do the simple math of: we pay for x power, we have y customers. x / y would be a rough but probably pretty accurate number if we are talking tens of thousands to millions of customers.
silence7@slrpnk.net 6 months ago
You can produce a remarkably good estimate by looking at CPU and GPU utilization out of procfs and profiling a handful of similar machines power use with similar utilization and workloads.
Network is less than 5% of power use for non-GPU loads; probably less for GPU.
DarkDarkHouse@lemmy.sdf.org 6 months ago
Sure, you can do that at an aggregate level, but then how do you divide it by customer? And even then, some setups will be more efficient than others, so you’d only get that setup’s usage.
And even if you do that and can narrow it down to a single user and a single prompt, you can still only roughly predict how long it will think and how long the response will be.