Comment on Selfhosting Sunday! What's up?
irmadlad@lemmy.world 1 day ago LOL Not quite, however, I am choking on the 5090 but as I understand, that’s the better option, and 32 GB seems to be the sweetspot for AI.
irmadlad@lemmy.world 1 day ago
brucethemoose@lemmy.world 1 day ago
First of all, I mean zero offense with any purchase decision. A 5090 is very good.
…But if I were paying that kind of money, I’d probably get a 4090 and a new motherboard/CPU instead. Maybe a used DRR4 threadripper system.
Hybrid (CPU + GPU) inference is where it’s at these days. It opens up a whole world of huge MoE models. And having a big CPU is much more important for that than having a 5090 instead of a 4090 or something.
Why you do this to me? LOL Injecting your options. But seriously thank you for the advice. I’m really green in the AI arena. So I’m trying to feel my way around, trying not to spend money on equipment I’ll regret later.