Yeah… You do it, but do you think the UK government does?
Comment on UK government trial of Microsoft's M365 Copilot finds no clear productivity boost
sugar_in_your_tea@sh.itjust.works 2 days agoIt doesn’t have to, you can run LLMs locally. We do at my org, and we only have a few dozen people using it, and it’s running on relatively modest hardware (Mac Mini for smaller models, Mac Studio for larger models).
themurphy@lemmy.ml 2 days ago
sugar_in_your_tea@sh.itjust.works 2 days ago
My point is they could if they were concerned about data leaks.
squaresinger@lemmy.world 2 days ago
Yeah, shitty toy ones. This here is about productivity, not about a hobby. And not even real state-of-the-art models were able to actually give a productivity advantage.
sugar_in_your_tea@sh.itjust.works 1 day ago
Our self-hosted ones are quite good and get the job done. We use them a lot for research, and it seems to do a better job than most search engines. We also link it to internal docs and it works pretty well for that too.
If you run a smaller model at home because you have limited RAM, yeah, you’ll have less effective models. We can’t run the top models on our hardware, but we can run much larger models than most hobbyists. We’ve compared against the larger commercial models, and they work well, if little slowly.