Comment on Google quietly released an app that lets you download and run AI models locally

AmbiguousProps@lemmy.today ⁨1⁩ ⁨month⁩ ago

• Greg@lemmy.ca ⁨1⁩ ⁨month⁩ ago

  Ollama can’t run on Android

  source

  • AmbiguousProps@lemmy.today ⁨1⁩ ⁨month⁩ ago

    That’s fair, but I think I’d rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.

    source

    • Greg@lemmy.ca ⁨1⁩ ⁨month⁩ ago

      Yes, that’s my setup. But this will be useful for cases where internet connection is not reliable

      source
    • OhVenus_Baby@lemmy.ml ⁨1⁩ ⁨month⁩ ago

      How is Ollama compared to GPT models? I used the paid tier for work and I’m curious how this stacks up.

      source

      • AmbiguousProps@lemmy.today ⁨1⁩ ⁨month⁩ ago

        It’s decent, with the deepseek model anyway. It’s not as fast and has a lower parameter count though. You might just need to try it and see if it fits your needs or not.

        source
  • Euphoma@lemmy.ml ⁨1⁩ ⁨month⁩ ago

    You can use it in termux

    source

    • Greg@lemmy.ca ⁨1⁩ ⁨month⁩ ago

      Has this actually been done? If so, I assume it would only be able to use the CPU

      source

      • Euphoma@lemmy.ml ⁨1⁩ ⁨month⁩ ago

        Yeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk

        source
  • Diplomjodler3@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Is there any useful model you can run on a phone?

    source
  • gens@programming.dev ⁨1⁩ ⁨month⁩ ago

    Llama.cpp (on which ollama runs on) can. And many chat programs for phones can use it.

    source
  • pirat@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Try PocketPal instead

    source