Comment on What type of computer setup would one need to run ai locally?
panda_abyss@lemmy.ca 1 day ago
High RAM for MOE models, high VRAM for dense models, and the highest GPU memory bandwidth you can get.
For stable diffusion models (comfyui), you want high VRAM and bandwidth. Diffusion is a GPU heavy and memory intensive operation.
Software/driver support is very important for diffusion models and comfy UI, so your best experience will be Nvidia cards.
I think realistically you need 80gb+ of RAM for things like qwen image quants (40 for model, 20-40 for LORA adapters in ComfyUI to get output).
I run an 128gb AMD AI 395+ Max rig, qwen image takes 5-20 minutes per 720p qwen image result in ComfyUI. Batching offers an improvement, reducing iterations during prototyping makes a huge difference. I have not tested since the fall though, and the newer models are more efficient.
ikidd@lemmy.world 1 day ago
Framework desktop?
panda_abyss@lemmy.ca 1 day ago
Yes
ikidd@lemmy.world 1 day ago
I’ve really been mulling one of those over with 128GB. I’m on Claude Max and Cerebras $50 so I’m using a good amount of $200/mo for coding and Openclaw. Is it worth it for light coding, or are you only doing SD with it?
panda_abyss@lemmy.ca 1 day ago
It would not be worth it as a replacement for Claude.
80% of my issue is that it’s AMD and their drivers are still awful. 20% is that the token generation speed very slow, especially compared to commercial models running on dedicated hardware. MOE models are fine, dense models are too slow for meaningful workflows. ComfyUI is decent, but I’m not seriously into image gen.
I have a lot of fun with it, but I have not been able to use it for any actual AI dev.