How is anyone going to run those powerful models locally if the hardware is unaffordable/unobtainable?
No one is going to pay for AI it because no one wants it to begin with.
How is anyone going to run those powerful models locally if the hardware is unaffordable/unobtainable?
No one is going to pay for AI it because no one wants it to begin with.
jj4211@lemmy.world 2 weeks ago
In this scenario, you have a number of these AI companies contributing to the hoarding having their equipment handled through ‘asset recovery’, which means that at least companies that can drive 15kw to a system and water cooling will probably get them on the cheap and run it on their premise, or in a colo. Maybe some of those parts will trickle down, but admittedly a good chunk of the stuff is hard to accommodate in a residential setting.
Longer term, the hardware becomes obtainable as supply chains re-calibrate back to identcal or more close solutions. Ten years ago, a datacenter GPU was likely to be same hardware as consumer, but with a different thermal solution, firmware, and the video ports unpopulated. The AI rush has made them shift to exotic packaging so they can have absurdly unreasonable wattage in small places that doesn’t work in home settings. I anticipate a swing back that way eventually.