barebones 780m amd mini pc. 128gb ddr5600. About $700. Under $800. Vulkan is good, non AMD supported version of ROCm works for some, because AMD says fuck you to its users. $2000 higher end mini pc with 8600s gpu is other option for high ram with under double the performance.
Comment on THE NVIDIA AI GPU BLACK MARKET | Investigating Smuggling, Corruption, & Governments
PalmTreeIsBestTree@lemmy.world 1 week agoYou could also buy the Apple studio with its large amount of unified ram for a similar price of a 5090.
humanspiral@lemmy.ca 1 week ago
brucethemoose@lemmy.world 1 week ago
The pricing for memory is still pretty bad. $4K for 96GB, $5.6K for 256GB, $10K for 512GB. One can get 128GB on the M4 Max for $3.5K, at the cost of a narrower bus so it’s even slower.
SOTA quantization for these is mostly DIY. There aren’t many MLX DWQs or trellis-quantized GGUFs floating around.
And if you want to finetune or tinker instead of just run, you’re at an enormous disadvantage there.