Comment on Chatbots Make Terrible Doctors, New Study Finds
SuspciousCarrot78@lemmy.world 6 days agoDepends which bit you mean specifically.
The “router” side is a offshoot of a personal project. It’s python scripting and a few other tricks, such as JSON files etc. Full project details for that here
github.com/BobbyLLM/llama-conductor
The tech stack itself:
- llama.cpp
- Qwen 2.5-1.5 GGUF base (by memory, 5 bit quant from HF Alibaba repository)
- The python router (more sophisticated version of above)
- Policy documents
- Front end (OWUI - may migrate to something simpler / more robust)
realitista@lemmus.org 6 days ago
Thanks it’s really interesting to see some real work applications and implementations of AI for practical workloads.
SuspciousCarrot78@lemmy.world 6 days ago
Very welcome :)
As it usually goes with these things, I built it for myself then realised it might have actual broader utility. We shall see!