Comment on Have We Reached Peak AI?

• CeeBee@lemmy.world ⁨8⁩ ⁨months⁩ ago

  they literally have no mechanism to do any of those things.

  What mechanism does it have for pattern recognition?

  that is literally how it works on a coding level.

  Neural networks aren’t “coded”.

  It’s called an LLM for a reason.

  That doesn’t mean what you think it does. Another word for language is communication. So you could just as easily call it a Large Communication Model.

  Neural networks have hundreds of thousands (at the minimum) of interconnected layers. Llama-2 has 76 billion parameters. The newly released Grok has over 300 billion. And though we don’t have official numbers, ChatGPT 4 is said to be close to a trillion.

  The interesting thing is that when you have neural networks of such a size and you feed large amounts of data into it, emergent properties start to show up. More than just “predicting the next word”, it starts to develop a relational understanding of certain words that you wouldn’t expect. It’s been shown that LLMs understand things like Miami and Houston are closer together than New York and Paris.

  Those kinds of things aren’t programmed, they are emergent from the dataset.

  As for things like creativity, they are absolutely creative. I have asked seemingly impossible questions (like a Harlequin story about the Terminator and Rambo) and the stuff it came up with was actually astounding.

  They regularly use tools. Lang Chain is a thing. There’s a new LLM called Devin that can program, look up docs online, and use a command line terminal. That’s using a tool.

  That also ties in with problem solving. Problem solving is actually one of the benchmarks that researchers use to evaluate LLMs. So they do problem solving.

  To problem solve requires the ability to do analysis. So that check mark is ticked off too.

  Just about anything that’s a neutral network can be called an AI, because the total is usually greater than the sun of parts.

  source