I’m pretty sure Google’s AI is fed by the same spider that goes out and finds every new or changed web page (or a variant of that).

As soon as someone writes an article about how AI gets something wrong and provides a solution, that solution is now in the AI’s training data.

OTOH, that means it’s probably also ingesting a lot of AI generated slop, which causes its own set of problems.