Comment

Comment on I'm looking for an article showing that LLMs don't know how they work internally

theunknownmuncher@lemmy.world ⁨5⁩ ⁨days⁩ ago

You’re confusing the confirmation that the LLM cannot explain it’s under-the-hood reasoning as text output, with a confirmation of not being able to reason at all. Anthropic is not claiming that it cannot reason. They actually find that it performs complex logic and behavior like planning ahead.

source

Sort:hotnew top

adespoton@lemmy.ca ⁨5⁩ ⁨days⁩ ago
No, they really don’t. It’s a large language model. Input cues instruct it as to which weighted path through the matrix to take. Those paths are complex enough that the human mind can’t hold all the branches and weights at the same time. But there’s no planning going on; the model can’t backtrack a few steps, consider different outcomes and run a meta analysis. Other reasoning models can do that, but not language models; language models are complex predictive translators.

source
- theunknownmuncher@lemmy.world ⁨5⁩ ⁨days⁩ ago
  
  To write the second line, the model had to satisfy two constraints at the same time: the need to rhyme (with “grab it”), and the need to make sense (why did he grab the carrot?). Our guess was that Claude was writing word-by-word without much forethought until the end of the line, where it would make sure to pick a word that rhymes. We therefore expected to see a circuit with parallel paths, one for ensuring the final word made sense, and one for ensuring it rhymes.
  
  Instead, we found that Claude plans ahead. Before starting the second line, it began “thinking” of potential on-topic words that would rhyme with “grab it”. Then, with these plans in mind, it writes a line to end with the planned word.
  
  🙃 actually read the research?
  
  source
  - glizzyguzzler@lemmy.blahaj.zone ⁨2⁩ ⁨days⁩ ago
    No, they’re right. The “research” is biased by the company that sells the product and wants to hype it. Many layers don’t make think or reason, but they’re glad to put them in quotes that they hope peeps will forget were there.
    
    source