Comment on What budget friendly GPU for local AI workloads should I aim for?
MTK@lemmy.world 6 days ago
Buying new: Basically all of the integrated memory units like macs and amd’s new AI chips, after that any modern (last 5 years) gpu while focusing only on vram (currently nvidia is more properly supported in SOME tools)
Buying second hand: not likely to find any of the integrated memory stuff, so any GPU from the last decade that is still officially supported and focusing on vram.
8gb is enough to run basic small models, 20+ for pretty capable 20-30b models, 50+ for the 70b ones and 100-200+ for full sized models.
These are rough estimates, do your own research as well.