Comment

Comment on I'm looking for an article showing that LLMs don't know how they work internally

theunknownmuncher@lemmy.world ⁨10⁩ ⁨months⁩ ago

It’s true that LLMs aren’t “aware” of what internal steps they are taking, so asking an LLM how they reasoned out an answer will just output text that statistically sounds right based on its training set, but to say something like “they can never reason” is provably false.

Its obvious that you have a bias and desperately want reality to confirm it, but there’s been significant research and progress in tracing internals of LLMs, that show logic, planning, and reasoning. Neural networks and very powerful, after all, you are one too. Can you reason?

source

Sort:hotnew top

ohwhatfollyisman@lemmy.world ⁨10⁩ ⁨months⁩ ago

but there’s been significant research and progress in tracing internals of LLMs, that show logic, planning, and reasoning.

would there be a source for such research?

source
- theunknownmuncher@lemmy.world ⁨10⁩ ⁨months⁩ ago
  anthropic.com/…/tracing-thoughts-language-model for one, the exact article OP was asking for
  
  source
  - ohwhatfollyisman@lemmy.world ⁨10⁩ ⁨months⁩ ago
    but this article espouses that llms do the opposite of logic, planning, and reasoning?
    
    quoting:
    
    Claude, on occasion, will give a plausible-sounding argument designed to agree with the user rather than to follow logical steps. We show this by asking it for help on a hard math problem while giving it an incorrect hint. We are able to “catch it in the act” as it makes up its fake reasoning,
    
    are there any sources which show that llms use logic, conduct planning, and reason (as was asserted in the 2nd level comment)?
    
    source
    theunknownmuncher@lemmy.world ⁨10⁩ ⁨months⁩ ago
    No, you’re misunderstanding the findings. It does show that LLMs do not explain their reasoning when asked, which makes sense and is expected. They do not have access to their inner-workings and generate a response that “sounds” right, but tracing their internal logic shows they operate differently than what they claim, when asked. You can’t ask an LLM to explain its own reasoning. But the article shows how they’ve made progress with tracing under-the-hood, and the surprising results they found about how it is able to do things like plan ahead, which defeats the misconception that it is just “autocomplete”
    
    source
glizzyguzzler@lemmy.blahaj.zone ⁨10⁩ ⁨months⁩ ago
Too deep on the AI propaganda there, it’s completing the next word. You can give the LLM base umpteen layers to make complicated connections, still ain’t thinking.

The LLM corpos trying to get nuclear plants to power their gigantic data centers while AAA devs aren’t trying to buy nuclear plants says that’s a straw man and you simultaneously also are wrong.

Using a pre-trained and memory-crushed LLM that can run on a small device won’t take up too much power. But that’s not what you’re thinking of. You’re thinking of the LLM only accessible via ChatGPT’s api that has a yuge context length and massive matrices that needs hilariously large amounts of RAM and compute power to execute. And it’s still a facsimile of thought.

It’s okay they suck and have very niche actual use cases - maybe it’ll get us to something better. But they ain’t gold, they ain’t smart, and they ain’t worth destroying the planet.

source
- theunknownmuncher@lemmy.world ⁨10⁩ ⁨months⁩ ago
  
  it’s completing the next word.
  
  Facts disagree, but you’ve decided to live in a reality that matches your biases despite real evidence, so whatever 👍
  
  source
  - glizzyguzzler@lemmy.blahaj.zone ⁨10⁩ ⁨months⁩ ago
    It’s literally tokens. Doesn’t matter if it completes the next word or next phrase, still completing the next most likely token 😎😎 can’t think can’t reason can witch’s brew facsimile of something done before
    
    source
    Epp2@lemmynsfw.com ⁨10⁩ ⁨months⁩ ago
    Why aren’t they tokens when you use them? Does your brain not also choose the most apt selection for the sequence to make maximal meaning in the context prompted? I assert that after a sufficiently complex obfuscation of the underlying mathematical calculations the concept of reasoning becomes an exercise in pedantic dissection of the mutual interpretation of meaning. Our own minds are objectively deterministic, but the obfuscation provided by lack of direct observation provides the quantum cover fire needed to claim we are not just LLM equivalent representation on biological circuit boards.
    
    source
A_Union_of_Kobolds@lemmy.world ⁨10⁩ ⁨months⁩ ago
[deleted]
source
- theunknownmuncher@lemmy.world ⁨10⁩ ⁨months⁩ ago
  ollama is not an LLM, but a program used to run them. What model are you running?
  
  source
  - A_Union_of_Kobolds@lemmy.world ⁨10⁩ ⁨months⁩ ago
    Yes I’m well aware thank you.
    
    Gemma3 was latest when I installed it.
    
    source
    theunknownmuncher@lemmy.world ⁨10⁩ ⁨months⁩ ago
    I’ve been very unimpressed by gemma3. 1b, 4b, 12b? 27b is probably your best chance at coherent results. Try qwen3:32b
    
    source