Comment on What exactly is a self-hosted small LLM actually good for (<= 3B)

hendrik@palaver.p3x.de ⁨2⁩ ⁨weeks⁩ ago

I think that's a size where it's a bit more than a good autocomplete. Could be part of a chain for retrieval augmented generation. Maybe some specific tasks. And there are small machine learning models that can do translation or sentiment analysis, though I don't think those are your regular LLM chatbots... And well, you can ask basic questions and write dialogue. Something like "What is an Alpaca?" will work. But they don't have much knowledge under 8B parameters and they regularly struggle to apply their knowledge to a given task at smaller sizes. At least that's my experience. They've become way better at smaller sizes during the last year or so. But they're very limited.

source
Sort:hotnewtop