SirMaple__

@SirMaple__@lemmy.ca

This is a remote user, information on this page may be incomplete. View original ↗

⁨Comment⁩ on ⁨Is anyone self-hosting their own AI?⁩ ⁨⁨1⁩ ⁨week⁩ ago⁩:
I have the following hardware

2 x ASUS GX10 128GB(each) 2 x Mac Mini M4 Pro 24GB(each)

The GX10s are clustered and run DeepSeek-V4-Flash-0731-Abliterated. The M4 Pros run various utility LLMs such as Qwen 3.8

I have the following software running

OpenWebUI Hermes Agent LiteLLM Llama.cpp vLLM

StoryBinder - custom web app that I use for story/script writing when in the mood

LASM - Local AI Stack Manger - custom web interface to manage pretty much all aspects of my local stack. Agents run on the 4 nodes as well as my NAS(Download GGUF/HF snapshots once and copy out to the nodes)

Have yet to connect my Home Assistant into the stack.
⁨Comment⁩ on ⁨Hedge fund billionaire says US may face ‘worse than a recession’ from Trump tariffs⁩ ⁨⁨1⁩ ⁨year⁩ ago⁩:
Should have thought of that when voting …