herseycokguzelolacak
@herseycokguzelolacak@lemmy.ml
This is a remote user, information on this page may be incomplete. View at Source ↗
- Comment on What exactly is a self-hosted small LLM actually good for (<= 3B) 1 day ago:
Not on top of my head, but there must be something. llama.cpp and vllm have basically solved the inference problem for LLMs. What you need is a RAG solution on top that also combines it with web search.
- Comment on What exactly is a self-hosted small LLM actually good for (<= 3B) 2 days ago:
for coding tasks you need web search and RAG. It’s not the size of the model that matters, since even the largest models find solutions online.