Comment on People Are Losing Loved Ones to AI-Fueled Spiritual Fantasies

• Zozano@aussie.zone ⁨4⁩ ⁨weeks⁩ ago

  No, I’m specifically describing what an LLM is. It’s a statistical model trained on token sequences to generate contextually appropriate outputs. That’s not “tools it uses,” that is the model. When I said it pattern-matches reasoning and identifies contradictions, I wasn’t talking about external plug-ins or retrieval tools, I meant the LLM’s own internal learned representation of language, logic, and discourse.

  You’re drawing a false distinction. When GPT flags contradictions, weighs claims, or mirrors structured reasoning, it’s not outsourcing that to some other tool, it’s doing what it was trained to do. It doesn’t need to understand truth like a human to model the structure of truthful argumentation, especially if the prompt constrains it toward epistemic rigor.

  Now, if you’re talking about things like code execution, search, or retrieval-augmented generation, then sure, those are tools it can use. But none of that was part of my argument. The ability to track coherence, cite counterexamples, or spot logical fallacies is all within the base LLM. That’s just weights and training.

  So unless your point is that LLMs aren’t humans, which is obvious and irrelevant, all you’ve done is attack your own straw man.

  source

  • dzso@lemmy.world ⁨4⁩ ⁨weeks⁩ ago

    So you’re describing a reasoning model, which is 1) still based on statistical token sequences and 2) trained on another tool (logic and discourse) that it uses to arrive at the truth. It’s a very fallible process. I can’t even begin to count the number of times that a reasoning model has given me a completely false conclusion. Research shows that even the most advanced LLMs are giving incorrect answers as much as 40% of the time IIRC. Which reminds me of a really common way that humans arrive at truth, which LLMs aren’t capable of:

    Fuck around and find out. Also known as the scientific method.

    source

    • Zozano@aussie.zone ⁨4⁩ ⁨weeks⁩ ago

      You’re not actually disagreeing with me, you’re just restating that the process is fallible. No argument there. All reasoning models are fallible, including humans. The difference is, LLMs are consistently fallible, in ways that can be measured, improved, and debugged (unlike humans, who are wildly inconsistent, emotionally reactive, and prone to motivated reasoning).

      Also, the fact that LLMs are “trained on tools like logic and discourse” isn’t a weakness. That’s how any system, including humans, learns to reason. We don’t emerge from the womb with innate logic, we absorb it from language, culture, and experience. You’re applying a double standard: fallibility invalidates the LLM, but not the human brain? Come on.

      And your appeal to “fuck around and find out” isn’t a disqualifier; it’s an opportunity. LLMs already assist in experiment design, hypothesis testing, and even simulating edge cases. They don’t run the scientific method independently (yet), but they absolutely enhance it.

      So again: no one’s saying LLMs are perfect. The claim is they’re useful in evaluating truth claims, often more so than unaided human intuition. That you’ve encountered hallucinations doesn’t negate that - it just proves the tool has limits, like every tool. The difference is, this one keeps getting better.

      source