Comment on Chatbots Make Terrible Doctors, New Study Finds

• SuspciousCarrot78@lemmy.world ⁨5⁩ ⁨weeks⁩ ago

  Depends which bit you mean specifically.

  The “router” side is a offshoot of a personal project. It’s python scripting and a few other tricks, such as JSON files etc. Full project details for that here

  github.com/BobbyLLM/llama-conductor

  The tech stack itself:

  • llama.cpp
  • Qwen 2.5-1.5 GGUF base (by memory, 5 bit quant from HF Alibaba repository)
  • The python router (more sophisticated version of above)
  • Policy documents
  • Front end (OWUI - may migrate to something simpler / more robust)

  source

  • realitista@lemmus.org ⁨5⁩ ⁨weeks⁩ ago

    Thanks it’s really interesting to see some real work applications and implementations of AI for practical workloads.

    source

    • SuspciousCarrot78@lemmy.world ⁨5⁩ ⁨weeks⁩ ago

      Very welcome :)

      As it usually goes with these things, I built it for myself then realised it might have actual broader utility. We shall see!

      source