Comment on Lowering power consumption on Opteron
grue@lemmy.world 2 weeks agoNothin’ I’m running, that’s for sure!
It’s not really that there are services that require that much processing power for a single request; it’s that it’s designed to handle normal requests for hundreds or thousands of users at once.
I suppose that supporting 0.5TB of RAM means it could deal with quite a big LLM, but any sort of halfway-modern GPU would absolutely run circles around it in terms of tokens per second, on any model that fit in their VRAM.
spaghettiwestern@sh.itjust.works 2 weeks ago
Sounds like my laptop will be plenty fast for some time to come.
This platform doesn’t use much power to begin with, but I do run TLP using a battery profile (despite the fact it’s always plugged in) to lower the power consumption a bit further. There’s no noticeable impact on application performance. If you’re running Linux maybe it will work on your hardware.