randomly doesn’t mean equiprobable. If you’re sampling a probability distribution, it’s random. Temperature 0 is never used, otherwise a lot of stuff would consistently hallucinate the exact same thing
Comment on A courts reporter wrote about a few trials. Then an AI decided he was actually the culprit.
theterrasque@infosec.pub 1 month agorandomly sampled.
Semi-randomly. There’s a lot of sampling strategies. For example temperature, top-K, top-p, min-p, mirostat, repetition penalty…
vrighter@discuss.tchncs.de 1 month ago
theterrasque@infosec.pub 1 month ago
Temperature 0 is never used
It is in some cases, where you want a deterministic / “best” response. Seen it used in benchmarks, or when doing some “Is this comment X?” where X is positive, negative, spam, and so on. You don’t want the model to get creative there, but rather answer consistently and always the most likely path.
futatorius@lemm.ee 1 month ago
A more correct term is constrained randomness. You’re still looking at probability distribution functions, but they’re more complex than just a throw of the dice.