Is it? Is random variance the source of all hallucinations? I think it’s not; it’s more the fact that they don’t understand what they’re generating, they’re just looking for the most statistically probable next character.
Yeah, they aren’t trained to make “correct” responses, but reasonably looking responses; they aren’t truth systems. However, I’m not sure what a truth system would even look like. At a certain point truth/fact become subjective, meaning that we probably have a fundamental problem with how we think about and evaluate these systems.
I mean, it’s the whole reason programming languages were created, natural language is ambiguous.
Yeah, solipsism existing drives the point about truth home. Thing is, LLMs outright lie without knowing they’re lying, because there’s no understanding there. It’s statistics at the character level.
AI is not my field, so I don’t know, either.
spankmonkey@lemmy.world 3 weeks ago
If there are 800 sentences/whatever chunk of information it uses about what color a ball is, using the average can result in that sentence using red when it should be blue based on the current question or it could add information about balls that are a different type because it doesn’t understand what kind of ball it is talking about. It might be randomness, it might be using an average, or a combination of both.
Like if asked about ‘what color is a basketball’ and the training set includes a it of custom color combinations by each team it might return a combination of colors that doesn’t match a team like brown (default leather) and yellow. This could also be the answer if you asked for an example of a basketball that matched team colors, because it might keep the default color from a ball that just has a team logo.
If someone doesn’t know the training set it would probably look like it made something ip. To someone who knows it is impossible to tell of it is random, due to a lack of knowing what it is talking about, or if it had some other less obvious connection that combines the two which lead to yellow and brown result.