He notes that LLM vendors have been training their models on Wikipedia content. But if the content contains incorrect information and citations, you get the sort of circular (incorrect) reference that leads to misinformation.

One irony, he says, is that LLM vendors are now willing to pay for training data unpolluted by the hallucinated output their own products generate.