Efficiency still matters very much when self hosting. You need to consider power usage (do you have enough amps in your service to power a single GPU? probably. what about 10? probably not) and heat (it’s going to make you need to run more A/C in the summer, do you have enough in your service to power an A/C and your massive amount of GPUs? not likely).
Homes are not designed for huge amounts of hardware. I think a lot of self hosters (including my past self) can forget that in their excitement of their hobby. Personally, I’m just fine not running huge models at home. I can get by with models that can run on a single GPU, and even if I had more GPUs in my server, I don’t think the results (which would still contain many hallucinations) would be worth the power cost, strain on my A/C, and possible electrical overload.
atzanteol@sh.itjust.works 3 days ago
😑
TheMightyCat@ani.social 3 days ago
While I would still say it’s excessive to respond with “😑” i was too quick in waving these issues away.
Another commenter explained that residential power physically does not suppply enough to match high end gpus is why even for selfhosters they could be worth it.
TheMightyCat@ani.social 3 days ago
?