That’s essentially the thing that makes LLMs as unreliable as they are in everything else; they run on probabilities that have no anchor in reality. The game is just another contained reality to which the model has no direct connection.