Comment on The line between what is ai and what is programming will be very blurred in the future
howrar@lemmy.ca 4 weeks agoLLMs are necessarily non-deterministic,
There’s nothing about LLMs that force them to be non-deterministic. Given the same prompts and context, they will always output the same probability distribution. It’s then up to you what you decide to do with that distribution. If you decide to always choose the most likely output, then the entire thing will be deterministic. We just don’t do that because it’s less useful than stochastic output.
PeriodicallyPedantic@lemmy.ca 4 weeks ago
Yes, that’s why it’s necessary for it to be non-deterministic. Without non-determinism, there is no error recovery if it chooses the wrong token somewhere in the middle of the completion. That’s much less useful.
howrar@lemmy.ca 4 weeks ago
What’s this error recovery business you speak of?
PeriodicallyPedantic@lemmy.ca 4 weeks ago
So you can try your prompt again. Also to avoid getting stuck in loops of repeated text. And if your temperature is low, the LLM won’t produce very novel results.