anamethatisnt

Depending on the use and the size of your graphics card you might be better of trying Gemma 4 12B too.
On my Nvidia 5060 Ti 16GB I can use either Gemma 4 A4B IQ3_M (larger model but highly compressed) or Gemma 4 12B Q8_0 (smaller model but much less compression) and after a night of testing with the Gemma 4 12B Q8_0 I’d say it’s a better helper when it comes to coding but a worse gm for my dnd inspired fantasy world roleplaying adventure.

I imagine your “setup a selfhosted environment” is closer to the first.

Easy setup would be to use koboldcpp + SillyTavern + Gemma 4 26B A4B GGUF of the largest quant your graphics card can fit together with your context.
Remember to setup SillyTavern to allow network connections and create a user and password, default installation is localhost only.
Set the temperature to 0.2 and make sure all the formatting templates inside Advanced Formatting are set to Gemma 4.

Then use either Gemma 4 itself or the free google.com AI to create some “W++ Denze with Horizontal Lines summaries for Gemma 4” of the relevant documentation of the latest version of the home server apps your friends are gonna install.
You should ensure the resulting lorebook entries are no more than 1k tokens each to leave some context for your friends chats, add it to a character card in SillyTavern and set some keywords to allow them to load dynamically and not stay in context memory all the time.

Regarding the character card you can ask Gemma 4 to write that for you too, I find “Write a character card in W++ Denze with Horizontal Lines style for Gemma 4 with this name, personality, attitude and skillset” works well for that.
Then ask it to write a “First message prompt that starts with X, continues with Y and ends with Z for that character card” and you get a first draft to rewrite and paste into the “First Message” of the character card. The first message works as a template that Gemma 4 will imitate when you chat with it. Then simply try the chatbot out before letting others use it.

I find my own Gemma 4 26B A4B IQ3_M works well for practicing hiragana and katakana, discussing programming or troubleshoot existing code or writing a small function but it can’t be expected to write a correct DatabaseService.cs from scratch and stuff like that.

Oh and forget about finding good cards and lorebooks for SillyTavern use online, most users use it for NSFW Roleplaying chats. It is a very easy UI to use to create harnesses for your local LLM though.

Aphids when they molt and leave their old exoskeletons behind.
www.rhs.org.uk/biodiversity/aphids

While it is a bit of work to rename the files that would still be my recommendation. I find jellyfin to be a great match for my anime that way.

Image
Image

I find setting up an openvpn server with self-signed certificates + username and password login works well. You can even run it on tcp/443 instead of tcp/1194 if you want to make it less likely to be blocked.

Unless you wanna expose services to others my recommendation is always to hide your services behind a vpn connection.

I prefer running true vms too, but it is resource intensive.
Playing with lxcs and docker could allow one to run more services on a little beelink.

A VM is properly isolated and has it’s own OS and kernel. This improves security at the cost of overhead.
If you are starved for hardware resources then running lxcs instead of vms could give you more bang for the buck.

I remember trying both back when my server was new but missing something in dockge, can’t remember what right now.

I wouldn’t use RAID on USB.
If you only got 2x m.2 slots then I would probably prioritize disk space over RAID1 and ensure you got a backup up and running. There are m.2 to sata adapters but your Bee-link doesn’t have a suitable psu for that.

I would start with one VM running portainer and once that is up and running I would recommend learning how to backup and restore the VM. If you have enough disks I would look into ZFS RAID 1 for redundancy.
pve.proxmox.com/wiki/ZFS_on_Linux
Learning the redundancy and backup systems before having too many services active allows you to screw up and redo.