Even as a tool it lacks predictability / reproducability
If you use the same seed on the same model with the same weights you get the same results.
Even as a tool it lacks predictability / reproducability
If you use the same seed on the same model with the same weights you get the same results.
Nalivai@lemmy.world 1 day ago
That’s not the predictability we want. If I write a calculator that adds the output of rand() to any result, it will also be repeatable with the same seed on the same machine. It will be nin-functional as a calculator though.
Knock_Knock_Lemmy_In@lemmy.world 1 day ago
Depends on your use case. Adding 0.000001*rand() to a large number retains the functionality as a calculator.
Your argument that AI isn’t useful may be valid, but claiming that AI is not repeatable is false.
michaelmrose@lemmy.world 1 day ago
AI is in fact not meaningfully repeatable in actual usage patterns.
Knock_Knock_Lemmy_In@lemmy.world 23 hours ago
Agreed. The word “patterns” is an important qualification. LLMs are great for one off tasks, but not as part of a repeatable process.
yermaw@sh.itjust.works 16 hours ago
Technically its repeatable under perfect conditions? Probably I don’t understand it well enough.
Realistically repeatable when Average Joe gets his hands on it? Not today anyway.
Knock_Knock_Lemmy_In@lemmy.world 5 hours ago
The seed for the random number generator isn’t usually exposed to the user. Also the AI service providers regularly update the model and/or weights, producing different outputs.
If you are running your own LLM on your own hardware you can guarantee repeatablity, but that’s not what your average Joe does.