Comment on Microsoft seemingly just revealed that OpenAI lost $11.5B last quarter
CheeseNoodle@lemmy.world 19 hours agoI’ve been looking into local models lately, mostly out of vague paranoia that I should get one up and running before it becomes defacto illegal for normal people to own due to some kind of regulatory capture. Seems fairly doable at the momeny though not particularly user friendly.
sobchak@programming.dev 18 hours ago
They’re OK. It’s kinda amazing how much lossy data can be compressed into a 12GB model. They don’t really start to get comparable to the large frontier models until 70B+ parameters, and you would need serious hardware to run those.
CheeseNoodle@lemmy.world 14 hours ago
Oh I mean with setup, like I can download ollama and a basic model fine enough and get a generic chat bot. But if I want something that can scan through PDFs with direct citation (like the Nvidia LLM) or play a character then suddenly its all git repositiories and cringey youtube tutorials.
sobchak@programming.dev 12 hours ago
That would be nice. I don’t know how to do that other than programming it yourself with something like langgraph.