Comment on People Are Losing Loved Ones to AI-Fueled Spiritual Fantasies

• Zozano@aussie.zone ⁨4⁩ ⁨days⁩ ago

  I do understand what an LLM is. It’s a probabilistic model trained on massive corpora to predict the most likely next token given a context window. I know it’s not sentient and doesn’t “think,” and doesn’t have beliefs. That’s not in dispute.

  But none of that disqualifies it from being useful in evaluating truth claims. Evaluating truth isn’t about thinking in the human sense, it’s about pattern-matching valid reasoning, sourcing relevant evidence, and identifying contradictions or unsupported claims. LLMs do that very well, especially when prompted properly.

  Your insistence that this is “dangerous naïveté” confuses two very different things: trusting an LLM blindly, versus leveraging it with informed oversight. I’m not saying GPT magically knows truth, I’m saying it can be used as a tool in a truth-seeking process, just like search engines, logic textbooks, or scientific journals. None of those are conscious either, yet we use them to get closer to truth.

  If you’re worried about misuse, and so am I. But claiming the tool is inherently useless because it lacks consciousness is like saying microscopes can’t discover bacteria because they don’t know what they’re looking at.

  So again: if you believe GPT is inherently incapable of aiding in truth evaluation, the burden’s on you to propose a more effective tool that’s publicly accessible, scalable, and consistent. I’ll wait.

  source

  • dzso@lemmy.world ⁨4⁩ ⁨days⁩ ago

    What you’re describing is not an LLM, it’s tools that an LLM is programmed to use.

    source

    • Zozano@aussie.zone ⁨4⁩ ⁨days⁩ ago

      No, I’m specifically describing what an LLM is. It’s a statistical model trained on token sequences to generate contextually appropriate outputs. That’s not “tools it uses,” that is the model. When I said it pattern-matches reasoning and identifies contradictions, I wasn’t talking about external plug-ins or retrieval tools, I meant the LLM’s own internal learned representation of language, logic, and discourse.

      You’re drawing a false distinction. When GPT flags contradictions, weighs claims, or mirrors structured reasoning, it’s not outsourcing that to some other tool, it’s doing what it was trained to do. It doesn’t need to understand truth like a human to model the structure of truthful argumentation, especially if the prompt constrains it toward epistemic rigor.

      Now, if you’re talking about things like code execution, search, or retrieval-augmented generation, then sure, those are tools it can use. But none of that was part of my argument. The ability to track coherence, cite counterexamples, or spot logical fallacies is all within the base LLM. That’s just weights and training.

      So unless your point is that LLMs aren’t humans, which is obvious and irrelevant, all you’ve done is attack your own straw man.

      source

      • dzso@lemmy.world ⁨4⁩ ⁨days⁩ ago

        So you’re describing a reasoning model, which is 1) still based on statistical token sequences and 2) trained on another tool (logic and discourse) that it uses to arrive at the truth. It’s a very fallible process. I can’t even begin to count the number of times that a reasoning model has given me a completely false conclusion. Research shows that even the most advanced LLMs are giving incorrect answers as much as 40% of the time IIRC. Which reminds me of a really common way that humans arrive at truth, which LLMs aren’t capable of:

        Fuck around and find out. Also known as the scientific method.

        source

        • -> View More Comments