You can have applications where wall clock tine time is not all that critical but large model size is valuable, or where a model is very sparse, so does little computation relative to the size of the model, but for the major applications, I think that that’s correct.
Comment on GPU prices are coming to earth just as RAM costs shoot into the stratosphere - Ars Technica
NotMyOldRedditName@lemmy.world 4 hours agoIt does work, but it’s not really fast. I upgraded to 96gb from 32gb, and being able to play with the bigger models was fun, but it’s not something I could do anything productive with it was so slow.
tal@lemmy.today 4 hours ago
NotMyOldRedditName@lemmy.world 4 hours ago
Ya, that’s fair. If I was doing something I didn’t care about time on, it did work.
possiblylinux127@lemmy.zip 4 hours ago
Your bottle necked by memory bandwidth
You need ddr5 with lots of memory channels for it to he useful