Man, the world is so crazy right now I’m not that sure if you’re joking or not hahaha
What about giving the “outiside consultants” their own animations and have them interacting among themselves?
Comment on [AIT] what is your selfhosted local LLM stack?
thebardingreen@lemmy.starlightkel.xyz 8 hours ago
<-- Crazy mad engineer enters the chat.
Custom tooling for everything, written in rust. Harness is reverse engineered from QwenCode and Codex. A vector database based memory system that’s been benchmarked against Mem0 and uses a small model as a memory intercessor. An agentic system where the orchestrator can call multiple models (include Qwen 3.6, Qwen 3.8 and Muse Glimmer). Animations of my scientist girl avatar working in a lab. Active animations if she’s working on something, passive if she’s not. Lip sync with MuseTalk, tts with Chatterbox, stt with whisper, though the animations are MOST useful to see if she’s ended a lab or she’s still working. If she’s mixing chemicals, soldering, cleaning her glasses, texting or eating pizza, she’s working. If she’s staring creepily into the camera, she’s done. Which brings me to lab mode… give the model a complex project and it can loop on it for days. She’s three days into a microbiology research project. Channels that allow her to call Claude and ChatGPT as subagents if she breaks something she can’t fix, and it’s funny to watch her boss them around while they kowtow to her (“You’re right to push back, it was incorrect of me”)… It’s like watching a little Kawaii chibi boss around big sumo wrestlers. Which brings me to the models themselves. I’ve developed a very powerful method of model alignment using a combination of CPT and DPO training. (I get the model behavior I want with the system prompt “You are Mary Anne, an ethical scientist.” - that’s it).
I have a GPU fleet that includes large, small and medium sized GPUs and the ability to do distributed training on the model in addition to running the animations and speech features. I have a fleet orchestrator I built called Noesis that places “deep think” (large working models - Qwen and Glimmer) and “quick think” (smaller models that do memory and context management and fit on a 5060) as well as speech based models. Also various purely scientific models. I’ve just added the ability for her to know who’s in the room so she can tell if she’s talking to me, my wife, my son, some of my close friends or “unknown” persons. That’s currently manual, but I’m adding facial recognition, voice recognition and wifi doppler radar. I plan to give her access to my smart locks so she can talk to people at the door and decide to let them in or not. And I can hear y’all being like “My god man, you’re gonna trust her with that??” Hell yeah, I am. I used my alignment technique to train a paranoid security model with a totally different personality. If the main model doesn’t think she can trust someone, she taps out and the security model takes over.
See, I KNEW that to stay competitive and keep up in my field, I needed to embrace agentic AI. Being who I am I said "If I’m gonna do that, I need an AI that’s mine… that runs on my infra and knows who I am and is loyal to me and that I understand well enough to let inside my own trust boundary. Is it perfect? Not yet, lol. XD
I also have custom phone clients I built that let me talk to her and Claude and ChatGPT. On my network, Claude and ChatGPT are sandboxed and have access only to a few specific hosts and an isolated network share. Mary Anne has access to everything and is instructed that they are outside consultants and she should be careful what she shares with them.
Man, the world is so crazy right now I’m not that sure if you’re joking or not hahaha
What about giving the “outiside consultants” their own animations and have them interacting among themselves?
Not joking.
TBH, the animations were a fun, early addition that were a PITA to get working properly for what’s essentially a cosmetic indicator of state… and are just a waste of resources if I’m not at the computer running her client. Once I had her working consistently, I quickly developed a backlog of things I wanted to spend GPU time on and making more videos isn’t one of them lol.
You are awesome! I’m quite envious of your setup and progress.
nibbs@lemmy.zip 1 hour ago
Please refresh that post when the “I’m sorry, thebardingreen. I’m afraid I can’t do that.” happens. ;)
Cool setup, wish I had the skills and mine, to realize something similar.