Thanks for the tip. I have been looking into commenting out the ollama portion of the docker-compose.yml file and setting up llama.cpp instead for some time now. I suspect I don’t usually use features such as ollama pull to justify a fully-featured application.
Comment on Why is my home server using so much RAM for cache + buffer?
Shimitar@downonthestreet.eu 6 days ago
As a suggestion, ditch ollama and setup llama.cpp. it will work fine with openwebui and it’s much more efficient. (Unrelated to the ram/swap issue)
A9nWGzYt@lemmy.dbzer0.com 5 days ago
- Shimitar@downonthestreet.eu 5 days ago
Here is my experience wiki.gardiol.org/7-services/llamacpp
No ads no monetization just a wiki I keep for my future reference.
Never been happier than moving to llamacpp from ollama.
Can you talk more about this? What are you basing this on?
Llama CPP can run models offloading with CPU (es MoE models), you have much more control over how you run your models, and overall it’s very much actively developed.
For starters and people without too much willingness to mess up with stuff, ollama is a great choice. Llama.cpp gives you that extra power and flexibility that is so much worth for people who like to tweak and do more.
My personal opinion, of course. But based on having used both and ditched ollama for llama.cpp, so I am also biased, keep in mind.
But I will hardly go back to ollama now :)
slazer2au@lemmy.world 6 days ago
It has been a while since I looked but Ollama use to be llama.CCP with a cloud wrapper before they forked it and rewrote it. In the same way postman is curl with a cloud wrapper.
Tossing the extra bit will make it use less resources and support the actual project not someone forking oss, slapping on a paint job, rewriting a core function and still calling it oss.