ShellMonkey

My go to for messing with chat bots is Kobold that’ll let you split the work between multiple GPUs. I get the impression the actual processing is only done on one but it lets you load larger models with the extra memory.

I was using a Nvidia 3060 for a while, then had 2 in one box, then switched to a 3090.

The amount of vram is a big factor for decent performance. Getting it to not sound like a predictably repetitive bot though is a whole separate thing that is still kind of elusive.

I was probably using Emby already by then, had bought a lifetime license since it didn't require bouncing things off and outside server like Plex did (or was it that Plex was a renewing subscription, I forget) , so it just stayed out of inertia.

I can't say I've given Jellyfin a proper try (as in using it and the clients exclusively for a long period) but we have been using Emby for quite a while before I knew it existed.

If I'm not mistaken Jellyfin is actually a fork of Emby so they're pretty similar, but one is a bit older.