Comment on Self-Hosted AI is pretty darn cool
theterrasque@infosec.pub 3 months agoLlama3 8b can be run at 6gb vram, and it’s fairly competent. Gemma has a 9b I think, which would also be worth looking into.
Comment on Self-Hosted AI is pretty darn cool
theterrasque@infosec.pub 3 months agoLlama3 8b can be run at 6gb vram, and it’s fairly competent. Gemma has a 9b I think, which would also be worth looking into.