If there are 800 sentences/whatever chunk of information it uses about what color a ball is, using the average can result in that sentence using red when it should be blue based on the current question or it could add information about balls that are a different type because it doesn’t understand what kind of ball it is talking about. It might be randomness, it might be using an average, or a combination of both.

Like if asked about ‘what color is a basketball’ and the training set includes a it of custom color combinations by each team it might return a combination of colors that doesn’t match a team like brown (default leather) and yellow. This could also be the answer if you asked for an example of a basketball that matched team colors, because it might keep the default color from a ball that just has a team logo.

If someone doesn’t know the training set it would probably look like it made something ip. To someone who knows it is impossible to tell of it is random, due to a lack of knowing what it is talking about, or if it had some other less obvious connection that combines the two which lead to yellow and brown result.