Comment on What exactly is a self-hosted small LLM actually good for (<= 3B)

• herseycokguzelolacak@lemmy.ml ⁨1⁩ ⁨day⁩ ago

  Not on top of my head, but there must be something. llama.cpp and vllm have basically solved the inference problem for LLMs. What you need is a RAG solution on top that also combines it with web search.

  source