Comment on Running AI is so expensive that Amazon will probably charge you to use Alexa in future, says outgoing exec

• Zetta@mander.xyz ⁨9⁩ ⁨months⁩ ago

  Yes, and you can use run a language model like Pygmalion Al locally on koboldcpp and have a naughty AI chat as well. Or non sexual roleplay

  source

  • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

    Absolutely and there are many, many models that have iterated on and surpassed Pygmalion as well as loads of uncensored models specifically tuned for erotic chat. Steamy role play is one of the driving forces behind the rapid development of the technology on lower powered, local machines.

    source

    • Chreutz@lemmy.world ⁨9⁩ ⁨months⁩ ago

      Never underestimate human ingenuity

      When they’re horny

      source
    • das@lemellem.dasonic.xyz ⁨9⁩ ⁨months⁩ ago

      And where would one look for these sexy sexy AI models, so I can avoid them, of course…

      source

      • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

        Huggingface is where the models live. Anything that’s uncensored (and preferably based on llama 2) should work.

        Some popular suggestions at the moment might be HermesLimaRP 7B and Mythomax L2 13B for general roleplay that can easily include nsfw, or MLewdL2Chat 13B if you’re less concerned about plot development. All the links I provide here are from TheBloke on HF because they consistently have quality models available, but there are lots of talented people releasing models everyday tuned to assist with coding, translation, roleplay, general assistance (like chatgpt), writing, all kinds of things, really. Explore and try different models.

        General rule: if you don’t have a CPU, stick with 7B models. Otherwise, the bigger the better.

        source
    • Zetta@mander.xyz ⁨9⁩ ⁨months⁩ ago

      Which models do you think beat Pygmalion for erotic roleplay? Curious for research haha

      source

      • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

        Hey, I replied below, check it out.

        source

        • -> View More Comments
• MaxHardwood@lemmy.ca ⁨9⁩ ⁨months⁩ ago

  GPT4All is a neat way to run an AI chat bot on your local hardware.

  source

  • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

    Thanks for this, I haven’t tried GPT4All.

    Oobabooga is also very popular and relatively easy to run, but it’s not my first choice, personally.

    source

    • teuast@lemmy.ca ⁨9⁩ ⁨months⁩ ago

      it does have a very funny name though

      source
• teuast@lemmy.ca ⁨9⁩ ⁨months⁩ ago

  In five to ten years, imo, AI will be everywhere and may even replace the need for mobile Internet connections in terms of looking up information.

  You’re probably right, but I kinda hope you’re wrong.

  source

  • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

    Why?

    source

    • teuast@lemmy.ca ⁨9⁩ ⁨months⁩ ago

      Call it paranoia if you want. Mainly I don’t have faith in our economic system to deploy the technology in a way that doesn’t eviscerate the working class.

      source

      • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

        Oh, you are 100% justified in that! It’s terrifying, actually.

        But what I am envisioning is using small, open source models installed on our phones that can answer questions or just keep us company. These would be completely private, controlled by the user only, and require no internet connection. We are already very close to this reality, local AI models can be run on Android phones, but the small models that are best for phones are still pretty stupid (for now).

        Of course, living in our current Capitalist hellscape, it’s hard not to imagine that going awry to the point where we’ll all ‘rent’ AI from some asshole who censors it for our own ‘protection’ or puts ads in there somehow. But I guess I’m a dreamer.

        source
• scarabic@lemmy.world ⁨9⁩ ⁨months⁩ ago

  Don’t these models require rather a lot of storage?

  source

  • Soundhole@lemm.ee ⁨9⁩ ⁨months⁩ ago

    13B quantized models, generally the most popular for home computers with dedicated gpus, are between 6 and 10 gigs each. 7B models are between 3 and 6. So, no, not really?

    It is relative so, I guess if you’re comparing that to an atari 2600 cartridge then, yeah, it’s hella huge. But you can store multiple models for the same space cost as a single modern video game.

    source

    • scarabic@lemmy.world ⁨9⁩ ⁨months⁩ ago

      Yeah that’s not a lot. I mean… the average consumer probably has 10GB free on their boot volume.

      It is a lot to download. If we’re talking about ordinary consumers. Not unheard of though - some games on Steam are 50GB+

      So okay, storage is not prohibitive.

      source
  • art@lemmy.world ⁨9⁩ ⁨months⁩ ago

    Storage is getting cheaper every day and the models are getting smaller with the same amount of data.

    source

    • scarabic@lemmy.world ⁨9⁩ ⁨months⁩ ago

      I’m just curious - do you know what kind of storage is required?

      source