Comment on What exactly is a self-hosted small LLM actually good for (<= 3B)
MTK@lemmy.world 4 days ago
Have you tried RAG? I believe that they are actually pretty good for searching and compiling content from RAG.
So in theory you could have it connect to all of you local documents and use it for quick questions. Or maybe connected to your signal/whatsapp/sms chat history to ask questions about past conversations
catty@lemmy.world 4 days ago
No, what is it? How do I try it?
MTK@lemmy.world 4 days ago
RAG is basically like telling an LLM “look here for more info before you answer” so it can check out local documents to give an answer that is more relevant to you.
You just search “open web ui rag” and find plenty kf explanations and tutorials
iii@mander.xyz 4 days ago
I think RAG will be surpassed by LLMs with tool calling (aka agents), with search being one of the tools.
interdimensionalmeme@lemmy.ml 4 days ago
LLMs that train LoRas on the fly then query themselves with the LoRa applied