Comment on Seems legit
DarkCloud@lemmy.world 3 weeks ago
You can get offline versions of LLMs.
utopianfiat@lemmy.world 3 weeks ago
linkinkampf19@lemmy.world 3 weeks ago
sp3ctr4l@lemmy.dbzer0.com 3 weeks ago
I’ve been toying with Qwen3.
Opensource too!
JustAnotherKay@lemmy.world 2 weeks ago
Oh my god I feel so stupid. I’ve been arguing back and forth whether it was worth de-atomizing my steam deck to spin up alpaca in docker. I forgot they have a flatpak
sp3ctr4l@lemmy.dbzer0.com 2 weeks ago
Bazzite also has podman, though not specifically docker, in the core OS.
So… I have spun up one local LLM in Alpaca, told it what hardware, OS, and environment it is, told it to generate a context prompt to inform itself of all that… and its now helping me try to figure out how/if it is possible to set up a podman container/environment… for LLMs that either Alpaca does not yet support, or I am too stupid to figure out.
Alpaca even has tools. You can give an LLM the ability to search the web for something, and find some info or what not.
Presumably you could tell it to do a lot of things but that seems like a bad idea lol, anyway yeah, I was able to just tell it ‘go online and lookup bazzite, familiarize yourself with pertinent details, reformulate context prompt.’
Ghostalmedia@lemmy.world 3 weeks ago
I mean, most people have a local LLM in their pocket right now.
sp3ctr4l@lemmy.dbzer0.com 2 weeks ago
Unless I am missing something:
Most people do not have a local LLM in their pocket right now.
Most people have a client app that talks to a remote LLM, which ‘lives’ in an ecologically and economically dubious mega-datacenter, in their pocket right now.
GamingChairModel@lemmy.world 2 weeks ago
Plenty of the AI functions on phones are on-device. I know the iPhone is capable of several text-based processing (summarizing, translating) offline, and they have an API for third party developers to use on-device models. And the Pixels have Gemini Nano on-device for certain offline functions.
tetris11@feddit.uk 2 weeks ago
My phone does speech-to-text flawlessly offline, it’s a crazy useful little LLM tool
sp3ctr4l@lemmy.dbzer0.com 2 weeks ago
Oh!
Well, I didn’t know that.
I’m too poor to be able to afford such fancy phones.
Ghostalmedia@lemmy.world 2 weeks ago
Gemini nano, Apple Intelligence On-device, etc.
SubArcticTundra@lemmy.ml 3 weeks ago
criss_cross@lemmy.world 3 weeks ago
And gpt-oss is an offline version of chatgpt