Comment on Study finds that Chat GPT will cheat when given the opportunity and lie to cover it up later.

• DarkGamer@kbin.social ⁨9⁩ ⁨months⁩ ago

  They did no such thing. LLMs are probabilistic, not deterministic, and it can generate meaningful responses (to us) that the engineers neither predicted nor designed for.

  source

  • CrayonRosary@lemmy.world ⁨9⁩ ⁨months⁩ ago

    I get what you’re trying to say, but they are absolutely deterministic. All traditional (i.e., non quantum) computers and their programs are deterministic. Computation would be otherwise impossible. LLMs use a “random” seed value when generating their responses in order to “randomize” their responses, but it’s all perfectly deterministic. The same input plus the same seed results in the exact same response.

    Computers are just a series of binary switches, and programs and data are a bunch of instructions on how to initially set those switches before running a cycle of the CPU. It’s deterministic at every step.

    I put “random” in quotes because random number generators in software are also deterministic. They also use seed values (like the current time and the MAC address of the PC’s network interface) to generate numbers that only seem random. When true randomness is needed, a physical source of entropy must be used like an atmospheric sampler.

    The quirks of behavior you’re talking about have nothing to do with randomness vs determinism. Their behavior comes from the fact that their data sources are extremely large, and the neural network that it runs on was not designed by a human with specific behaviors like most algorithms are. The weights of the nodes in the neural network were generated by training and not by programmers, and it’s extremely complex, so no one can predict its output before running it.

    Of course, this is true of even basic algorithms a lot of the time.

    source

    • DarkGamer@kbin.social ⁨9⁩ ⁨months⁩ ago

      They also use seed values (like the current time and the MAC address of the PC’s network interface) to generate numbers that only seem random.

      For purposes of this discussion pseudo random with weights is probabilistic, or so close to it that this distinction is irrelevant.

      source