Comment on Why doesn't Nvidia have more competition?
brucethemoose@lemmy.world 3 weeks agoWRT pricing, I’m pretty sure AMD is typically a fraction of the price of Nvidia hardware on the enterprise side
I’m not as sure about this, but seems like AMD is taking a fat margin on the MI300X (and its sucessor?), and kinda ignoring the performance penalty. It’s easy to say “build it yourself!” but the reality is very few can, or will, do this, and will simply try to deploy vllm or vanilla TRL or something as best they can (and run into the same issues everyone does).
The ‘enthusiast’ side where all the tinkerer devs reside is totally screwed up though. AMD’s mirroring Nvidia’s VRAM cartel pricing when they have absolutely no reason to. It’s completely bonkers. AMD would be in a totally different place right now if they had sold 40GB/48GB 7900s for an extra $100 or $200.
The biggest culprit from what I can gather is that AMD’s GPU firmware/software side is basically still ATI camped up in Markham, divorced from the rest of the company in Austin that is doing great work with their CPU-side.
Yeah, it does seem divorced from the CPU division. But a lot of the badness comes from business decisions, even when the silicon is quite good, and some of that must be from Austin.
Glitchvid@lemmy.world 3 weeks ago
Eh, the biggest issue here is that most (post-secondary) students probably just have a laptop for whatever small GPGPU learning they’re doing, which is overwhelmingly dominated by Nvidia. For grad students they’ll have access to the institution resources, which is also dominated by Nvidia (this has been a concerted effort).
Only a few that explicitly pursue AMD hardware will end up with it, but that also requires significant foundational work for the effort. So the easiest path for research is throw students at CUDA and Nvidia hardware.
Basically, Nvidia has entrenched itself in the research/educational space, and that space is slow moving (Java is still the de facto CS standard, with only slow movements to Python happening at some universities), so I don’t see much changing, unless AMD decides it’s very hungry and wants to chase the market.
brucethemoose@lemmy.world 3 weeks ago
I dunno. From my more isolated perspective on GitHub and small LLM testing circles, I see a lot of 3090s, 4090s, sometimes arrays of 3060s/3090s or old P40s or MI50s, which people got basically for the purpose of experimentation and development because they can’t drop (or at least justify) $5K.
They would 100% drop that money on a 7900 48GB instead (as the sheer capacity is worth it over the speed hit and finickiness), and then do a whole bunch of bugfixing/testing on them. I know I would. Hence the Framework Strix Halo thing is sold out even though it’s… rather compute-lite compared to a 3090+ GPU.
It seems like a tiny market, but a lot of the frameworks/features/models being developed by humble open source devs filter up to the enterprise space.