I’m sure that’s true, but it is also noteworthy that any and every consideration that goes into the initial citation of the date before it is fed into the model introduces intended and unintended consequences on on the training.

Furthermore, the proliferation of the LLMs themselves is putting negative pressure on survival of the places where all the good data is sourced from in the first place. When traffic to a place lime stackoverflow is way down because everyone’s reading LLM answers (that the LLM training dataset got from stack overflow), there are less good conversations on stackoverflow to read. Some of these sources of training data may even be caused to cease to exist entirely.