And I can give numbers for people without a graphics card as well. I’m not a gamer so I didn’t bother buying a GPU. It takes my computer 20-40 minutes to generate a single image. Trying to generate video will make Linux step in and kill the process with an Out of Memory error. Generating text works. It’ll “write” a bit slower than I can read. And it takes additional time upfront to ingest my question.
iceberg314@slrpnk.net 1 week ago
I think it’s at least way more ethical to use locally hosted open weight AI. Mostly because a lot of these models are usually fine tuned and distilled to run on a single GPU and that is the direction the technology needs to develop to be sustainable.
Obviously for most things it’s way more ethical to just not use AI or LLMs.
I think people should at least experiment with running local models on their own hardware. It really puts it into perspective how much compute they use. Like the power, speed, and performance ratios of different size LLM and how absurd anything involving generating video is. I have a pretty beefy gaming PC and it took it like 6 minutes to generate 10 seconds of 720p video. And then that time goes up exponentially past 10 seconds. Like 20 minutes for a 15 second video
Running on a small model is basically like playing a modern full sized video game. When I measure my PC with a power meter
hendrik@palaver.p3x.de 1 week ago
mojofrododojo@lemmy.world 6 days ago
show me the one that’s built without theft and I might consider it.