Comment on Is anyone self-hosting their own AI?
anamethatisnt@sopuli.xyz 5 days agoGemma 4 12b unified as a q6 gguf fits with about 24k context inside your vram. koboldcpp+sillytavern works well on the software side.
Comment on Is anyone self-hosting their own AI?
anamethatisnt@sopuli.xyz 5 days agoGemma 4 12b unified as a q6 gguf fits with about 24k context inside your vram. koboldcpp+sillytavern works well on the software side.
laz@lemmy.dbzer0.com 5 days ago
Sc117