__matthew__

@__matthew__@lemmy.world

• ⁨Comment⁩ on ⁨Let's Encrypt is 10 years old today !⁩ ⁨⁨4⁩ ⁨days⁩ ago⁩:

  Lol I instinctively freaked out when I saw the post preview assuming it was going to be a post about a major data breach or exploit of some sort relating to Let’s Encrypt.

  I probably need more positivity in my life 😂

• ⁨Comment⁩ on ⁨Mozilla lays off 60 people, wants to build AI into Firefox⁩ ⁨⁨9⁩ ⁨months⁩ ago⁩:

  Sorry but has anyone in this thread actually tried running local LLMs on CPU? You can easily run a 7B model at varying levels of quantization (ie. 5 bit quantization) and get a generalized prompt-able LLM. Yeah, of course it’s going to take ~4GB of RAM (which is mem-mapped and paged into memory), but you can easily fine tune smaller more specific models (like the translation one mentioned above) and have surprising intelligence at a fraction of the resources.

  Take, for example, phi-2 which performs as well as 13B param models but with 2.7B params. Yeah, that’s still going to take 1.5GB RAM which Firefox wouldn’t reasonably ship, but many lighter weight specialized tasks could easily use something like a fine tuned 0.3B model with quantization.