I’ve been doing that, I use kubernetes, so kagent. Sammy at first but it calmed down. It’s neat to have it auto triage something that went down or failed, parse the logs, look at the hardware and health and then tell me when it’s time.

GPUs are sparse but I run on a 3090, VRAM is king. If you have one for a server it’s not horrible to spin up an llm