greybeard

Little eMudhra was asked to write down his a secret and hand it in to the teacher. He accidently gave the teacher the wrong paper.

I recently gave Strata a shot for my local AI. I’m very impressed with the performance they have eeked out of my hardware. I’ve got a rather beefy gaming desktop I built as an AI server back when the current craze started, before prices started going up, but models up until this point really failed to take full advantage of that hardware. Strata (which uses a Qwen model) both simplified the setup process, and it maxes out the hardware when running.

Strata acts like an OpenAI compatible API server, I then have Hermes as a “harness” to make an “agent”, which I interact with on “buzz”, which is like discord, but made for bots. Although Hermes also supports discord and several other chat platforms.

If you have the hardware to run Strata, I recommend setting it up, it comes with a basic chat interface, which you can then grill to assist you with setting up Hermes and whatever else.

One thing I didn’t mention, is security. I highly recommend you treat AI as unpredictable, unreliable, and capable of being taken over by bad actors. This isn’t a problem for a pure chat interface, but Hermes gives the AI tools to do things, and those tools could be used to take over your computer, scan your network, and create all kinds of havoc. For my part, Hermes is running in a virtual machine which is on an isolated network. I keep tight control on what it does and doesn’t have access to in my network main network. That may not be something you have the hardware or knowledge to do, at the very least I recommend running Hermes in a docker container, and not giving it access to your personal desktop nor credentials to things like your email account.

I’ve got a Makerbot Thing-O-Matic hanging around still. I’ve considered trying to get it running again, but it ran pretty terrible when new. Modern slicing might help a little, but I doubt it will save it from its own horrendously out of date design.

I think most of the comments misunderstood you, or I did.

Keeping a dedicated admin separate from your daily use user is best practice. In a business setting it is a critical security line, and also seriously helps with auditing and logging. It makes sure that if my computer gets compromised, the active cookies and session tokens are all a standard user. It’s not fool proof, but it really reduces risk.

That said, in my home lab, I don’t bother with that separation. It’s far more work than the effort justifies, when the worst result of your systems being compromised is your media library not being available. What you will want, in any case, is a “break glass” account into every system you setup to use SSO. A local administrator that isn’t tied to SSO available in case your IDP is down, certificates expire, or you accidentally paint yourself into a corner (and you will).

I recommend, if you don’t already, using a password manager like BitWarden and having every single local admin be a long, randomly generated password.

Qwen is what I’ve found to work the best for me so far. I haven’t done an exhaustive search or anything, but the mixture of experts 35b models seem to work pretty well.

I’ve got a little more vRAM to play with, 20GB. It still struggles with giving it enough context to be useful for agentic stuff.

My Christmas gift to family members last year were 3D printed lamps. I got a super cheap paper lantern style lamp from Ikea and removed the socket and cord, then printed a new base (filled with sand for weight), and printed a shade using a lithopane generator to have photos shine through. I was quite happy with the results.

If you know someone in IT, it’s worth asking them what their company does with old computers. At my last place, we sent off old PCs to recyclers, old often only being 3 years or so. Most of them were basic i5s with 16GB of RAM, but they were more than powerful enough for a good docker host. I’ve got small stack of them that I rescued from the recycle pile.

Not to mention the laptops that went to the recyclers. Of course, many of you might think that is wasteful, but the recyclers were refurbishing them and reselling. We got them off our hands for free, and they got to make a few bucks off them.