Comment on What's up, selfhosters? It's self hosting Sunday!

<- View Parent
Jakeroxs@sh.itjust.works ⁨2⁩ ⁨days⁩ ago

Interesting, I mainly have used text generation webui which has a search support plug in, kinda nifty to use my searxng instance for it. It’s a bit finicky though.

Another thing to keep in mind then (apologies if this is just repeating info you already know), you’d also want to keep in mind your total potential context size in relation to the model size, since both take up VRAM.

source
Sort:hotnewtop