Comment on Is anyone self-hosting their own AI?
anamethatisnt@sopuli.xyz 3 weeks agoGemma 4 12b unified as a q6 gguf fits with about 24k context inside your vram. koboldcpp+sillytavern works well on the software side.
Comment on Is anyone self-hosting their own AI?
anamethatisnt@sopuli.xyz 3 weeks agoGemma 4 12b unified as a q6 gguf fits with about 24k context inside your vram. koboldcpp+sillytavern works well on the software side.
laz@lemmy.dbzer0.com 3 weeks ago
Sc117