Comment on Squiggly Boie

vodka@feddit.org ⁨1⁩ ⁨month⁩ ago

• Aceticon@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

  Yeah, if you’re supposedly in AI/ML and don’t recognize a (stupidly simplified) diagram for a Neural Network, you don’t really make stuff with it, you’re just another user (probably a “prompt engineer”).

  Even people creating Machine Learning solutions with other techniques would recognize that as representing a Neural Network.

  source

  • NotANumber@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

    To be more specific this is an MLP (Multi-Layer Perceptron). Neural Network is a catch all term that includes other things such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks, Diffusion models and of course Transformers.

    What you are arguing online is some variant of a Generative Pre-Trained Transformer, which do have MLP or MoE layers but that’s only one part of what they are. They also have multi-headed attention mechanisms and embedding + unembedding vectors.

    I know all this and wouldn’t call myself a machine learning expert. I just use the things. Though I did once train a simple MLP like the one in the picture. I think it’s quite bad calling yourself a machine learning expert and not knowing all of this stuff and more.

    source

    • Aceticon@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

      Right, if I understood it correctly, what you see as “IF” is the multi-headed attention stuff.

      However the Genetic Algorithms stuff is something completelly different from Neural Networks: it’s basically an Evolutionary method of finding the best “formula” to process inputs to generate the desired output by assessing different variants of the “formula” with the training data, picking the best ones and then generating a new generation of variants from the best ones and assessing those and keep doing it until the error rate is below a certain value.

      As far as I can tell Genetic Algorithms can’t really scale to the size of something like an LLM (the training requirements would be even more insane) though that technique could be used to train part of a Neural Network or to create functional blocks that worked together with NNs.

      And yeah, MLPs trained via simple Backpropagation are exactly what I’m familiar with, having learned that stuff 3 decades ago as part of my degree when that was the pinnacle of NN technology and model architectures were still stupidly simple. That’s why I would be shocked if a so-called ML “expert” didn’t recognize that, as it’s the most basic form of Neural Network there is and it’s being doing the rounds for ages (that stuff was literally used to in automated postal code recognition in letters for automated mail sorting back in the 90s).

      I would expect that for people doing ML a simple MLP is as recognizable as binary is for programmers - sure people don’t work at that level anymore, but at they should at least recognize it.

      source

      • NotANumber@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

        Yes genetic algorithms are something different. Though they are used sometimes in training or arcitecting NNs, but not at the scale of modern LLMs.

        Fyi you can have all or nothing outputs from a perceptron or other network. It all depends on the activation function. Most LLMs don’t use that kind of activation function, but it is possible. Have you heard of bitnet? They use only one of three states for the neuron output in an LLM. It’s interesting stuff.

        source

        • -> View More Comments
      • NotANumber@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago

        Reading back over this I think you have me confused with another commentor. I don’t mention anything about IF in the commend you are replying to. Someone else did though.

        source
    • Holytimes@sh.itjust.works ⁨5⁩ ⁨weeks⁩ ago

      I can’t help but read MLP as my little pony and now I’m picturing you training a series of marshmallow horses to pretend to be human for the profits of our corporate overlords on social media.

      source
  • pineapplelover@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

    I looked up neural networks once and recognized this image. I’m on the low end of the dunning Kruger effect

    source

    • madjo@feddit.nl ⁨5⁩ ⁨weeks⁩ ago

      On which end of the curve? before or after the mount Dunning-Kruger?

      source

      • pineapplelover@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago

        The one who doesn’t know anything about neural networks

        source
• idiomaddict@lemmy.world ⁨1⁩ ⁨month⁩ ago

  I’m not a bot and I have no idea what the image is

  source

  • wander1236@sh.itjust.works ⁨1⁩ ⁨month⁩ ago

    It’s a diagram of a neural network which is the concept AI models are built on.

    source

    • idiomaddict@lemmy.world ⁨1⁩ ⁨month⁩ ago

      Thank you

      source
    • HeyThisIsntTheYMCA@lemmy.world ⁨1⁩ ⁨month⁩ ago

      LLM models or ML in general? i’ve only known folk who worked on ML, not actually done it myself

      source

      • kurwa@lemmy.world ⁨1⁩ ⁨month⁩ ago

        LLMs definitely, but not all machine learning uses neural networks

        source

        • -> View More Comments
      • NotANumber@lemmy.dbzer0.com ⁨1⁩ ⁨month⁩ ago

        Not all machine learning is AI. There are plenty of Machine Learning algorithms like Random Forests that are not neural networks. Deep learning would be big neural networks.

        source

        • -> View More Comments
    • ghost_of_faso3@lemmygrad.ml ⁨1⁩ ⁨month⁩ ago

      just from being a dumbass and not knowing what AI shit is I thought it was a representation of the double-slit experiment i

      source
  • IhaveCrabs111@lemmy.world ⁨1⁩ ⁨month⁩ ago

    That’s totally something a bot would say

    source