Comment

Comment on Anthropic has developed an AI 'brain scanner' to understand how LLMs work and it turns out the reason why chatbots are terrible at simple math and hallucinate is weirder than you thought

<- View Parent

Voroxpete@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

It really doesn’t. You’re just describing the “fancy” part of “fancy autocomplete.” No one was ever really suggesting that they only predict the next word. If that was the case they would just be autocomplete, nothing fancy about it.

What’s being conveyed by “fancy autocomplete” is that these models ultimately operate by combining the most statistically likely elements of their dataset, with some application of random noise. More noise creates more “creative” (meaning more random, less probable) outputs. They do not actually “think” as we understand thought. This can clearly be seen in the examples given in the article, especially to do with math. The model is throwing together elements that are statistically proximate to the prompt. It’s not actually applying a structured, logical method the way humans can be taught to.

source

Sort:hotnew top

FourWaveforms@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
Unfortunately, these articles are often written by people who don’t know enough to realize they’re missing important nuances.

source
- datalowe@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  It also doesn’t help that the AI companies deliberately use language to make their models seem more human-like and cogent. Saying that the model e.g. “thinks” in “conceptual spaces” is misleading imo. It abuses our innate tendency to anthropomorphize, which I guess is very fitting for a company with that name.
  
  On this point I can highly recommend this open access and even language-wise accessible article: link.springer.com/article/…/s10676-024-09775-5 (the authors also appear on an episode of the Better Offline podcast)
  
  source
  - FourWaveforms@lemm.ee ⁨3⁩ ⁨weeks⁩ ago
    I can’t contemplate whether LLMs think until someone tells me what it means to think. It’s too easy to rely on understanding the meaning of that word only through its typical use with other words.
    
    source
aesthelete@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
People are generally shit at understanding probabilities and even when they have a fairly strong math background tend to explain probablistic outcomes through anthropomorphism rather than doing the more difficult and “think-painy” statistical analysis that would be required to know if there was anything more to it.

I myself start to have thoughts that balatro is purposefully screwing me over or feeding me outcomes when it’s just randomness and probability as stated.

Ultimately, it’s easier (and more fun) for us to reason that way and it largely serves us better in everyday life.

But these things are entire casinos’ worth of probability and statistics in and of themselves, and the people developing them want desperately to believe that they are something more than pseudorandom probabilistic fancy autocomplete engines.

Add the difficulty of getting someone to understand how something works when their salary depends on them not understanding it to the existing inability of humans to reason probabilistically and the AGI from LLM delusion becomes near impossible to shake for some folks.

I wouldn’t be surprised if this AI hype bubble yields a cult in the end.

source
reev@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago
Genuine question regarding the rhyme thing, it can be argued that “predicting backwards isn’t very different” but you can’t attribute generating the rhyme first to noise, right? So how does it “know” (for lack of a better word) to generate the rhyme first?

source
- dustyData@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  It already knows which words are, statistically, more commonly rhymed with each other. From the massive list of training poems. This is what the massive data sets are for. One of the interesting things is that it’s not predicting backwards, exactly. It’s actually mathematically converging on the response text to the prompt, all the words at the same time.
  
  source
  - semperverus@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
    Which is exactly how we do it.
    
    source
    ThisIsNotHim@sopuli.xyz ⁨3⁩ ⁨weeks⁩ ago
    We also check to see if the word that popped into our heads actually rhymes by saying it out loud. Actual validation steps we can take is a bigger difference than being a little more robust.
    
    We also have non-list based methods like breaking the word down into smaller chunks to try to build up hopefully more novel rhymes. I imagine professionals have even more tools, given the complexity of more modern rhyme schemes.
    
    source