Comment on What exactly is a self-hosted small LLM actually good for (<= 3B)

<- View Parent
herseycokguzelolacak@lemmy.ml ⁨1⁩ ⁨day⁩ ago

Not on top of my head, but there must be something. llama.cpp and vllm have basically solved the inference problem for LLMs. What you need is a RAG solution on top that also combines it with web search.

source
Sort:hotnewtop