Comment on Anthropic has developed an AI 'brain scanner' to understand how LLMs work and it turns out the reason why chatbots are terrible at simple math and hallucinate is weirder than you thought

• Womble@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  Compared to a human who forms an abstract thought and then translates that thought into words. Which words I use has little to do with which other words I’ve used except to make sure I’m following the rules of grammar.

  Interesting that…

  Anthropic also found, among other things, that Claude “sometimes thinks in a conceptual space that is shared between languages, suggesting it has a kind of universal ‘language of thought’.”

  source

  • Carrolade@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    Yeah I caught that too, I’d be curious to know more about what specifically they meant by that.

    Being able to link all of the words that have a similar meaning, say, nearby, close, adjacent, proximal, side-by-side, etc and realize they all share something in common could be done in many ways. Some would require an abstract understanding of what spatial distance actually is, an understanding of physical reality. Others would not, one could simply make use of word adjacency, noticing that all of these words are frequently used alongside certain other words. This would not be abstract, it’d be more of a simple sum of clear correlations. You could call this mathematical framework a universal language if you wanted.

    Ultimately, a person learns meaning and then applies language to it. When I’m a baby I see my mother, and know my mother is something that exists. Then I learn the word “mother” and apply it to her. The abstract comes first. Can an LLM do something similar despite having never seen anything that isn’t a word or number?

    source

    • Womble@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      I don’t think that’s really a fair comparison, babies exist with images and sounds for over a year before they begin to learn language, so it would make sense that they begin to understand the world in non-linguistic terms and then apply language to that. LLMs only exist in relation to language so couldnt understand a concept separately to language, it would be like asking a person to conceptualise radio waves prior to having heard about them.

      source

      • Carrolade@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

        Exactly. It’s sort of like a massively scaled up example of the blind man and the elephant.

        source
    • aesthelete@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      Can an LLM do something similar despite having never seen anything that isn’t a word or number?

      No.

      source
  • MTK@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    Yeah but I think this is still the same, just not a single language. It might think in some mix of languages (which you can actuaysee sometimes if you push certain LLMs to their limit and they start producing mixed language responses.)

    But it still has limitations because of the structure in language. This is actually a thing that humans have as well, the limiting of abstract thought through internal monologue thinking

    source

    • Womble@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      Probably, given that LLMs only exist in the domain of language, still interesting that they seem to have a “conceptual” systems that is commonly shared between languages.

      source
  • TimewornTraveler@lemm.ee ⁨3⁩ ⁨weeks⁩ ago

    wow an AI researcher over hyping his own product. he’s just waxing poetic .

    we don’t even have a good sense of what thought IS, please tell Claude to call the philosophers because apparently he’s figured out consciousness

    source