Comment on Selfhosted & AI
midribbon_action@lemmy.blahaj.zone 21 hours agoLLMs are inextricably tied to nvidia gpus. Local or cloud, the technology exists to help the shovel salespeople. The gold diggers, everyone this tag is supposed to segregate, have been misled by corporations. Without their lies, and a pliant media, this tag would be unnecessary, and llms would be rolled out in a more limited and responsible way. To promote different uses of gold during a gold rush is going to inflate the bubble and enrich the rich, unless it is properly contextualized. Technology does not exist in a void, pretending it does digs us a deeper hole.
curbstickle@anarchist.nexus 20 hours ago
I run my local models on a Mac mini m2, but I could also be using AMD (ROCm), Intel with OpenVINO, or just CPU. Simple edge applications I could even use something like an RK3588.
Being tied to NVidia is marketing from NVidia, not the reality of LLMs.
midribbon_action@lemmy.blahaj.zone 20 hours ago
Congrats, you are in the 1%. And that is exactly the type of additional context I think is necessary when discussing it.
curbstickle@anarchist.nexus 19 hours ago
Sorry, I’m not following here. Are you suggesting I’m in the 1% of income earners? Because if so… LOL not even close, I’m barely in the top 40% by rough math.
Just to note, an apple silicon Mac is one of the most efficient (dollars, wattage, whatever take your pic) ways to run an LLM. Mine was a build target for iOS stuff for a client I’m now repurposing, that was a refurb that cost about $600.
I can’t even buy a GPU for that price these days.
midribbon_action@lemmy.blahaj.zone 19 hours ago
You are in the 1% of llm users.
Do you think anybody is choosing m2s over nvidia? Everybody running on cpu dreams of being rich enough to do real gold digging on an ampere. Amd is the only direct competitor, an order of magnitude smaller, and supports fewer models. I don’t think any hobbyist has one of these alternatives at the top of their wishlist, they are substitutes.