Comment on playing around with desktop ai

brucethemoose@lemmy.world ⁨4⁩ ⁨weeks⁩ ago

First, I would suggest using Deepseek 14B at least. The jump from 7B to 14B is huge, especially with the “bad” defaults ollama uses.

FYI you can utilize vram much more efficiently than ollama using exllama or base llama.cpp with a good quantization.

Secondly, Deepseek is good, but I would suggest trying “anti-slop” models that are trained to write better prose. My go to is EVA-Gutenberg since it’s trained on a base model.

source
Sort:hotnewtop