Comment on A courts reporter wrote about a few trials. Then an AI decided he was actually the culprit.
theterrasque@infosec.pub 2 days agorandomly sampled.
Semi-randomly. There’s a lot of sampling strategies. For example temperature, top-K, top-p, min-p, mirostat, repetition penalty…
vrighter@discuss.tchncs.de 2 days ago
randomly doesn’t mean equiprobable. If you’re sampling a probability distribution, it’s random. Temperature 0 is never used, otherwise a lot of stuff would consistently hallucinate the exact same thing
theterrasque@infosec.pub 2 days ago
It is in some cases, where you want a deterministic / “best” response. Seen it used in benchmarks, or when doing some “Is this comment X?” where X is positive, negative, spam, and so on. You don’t want the model to get creative there, but rather answer consistently and always the most likely path.