And why is that a problem?

Google searches also usually generate mostly useless results, which is impossible to combat. Thankfully the person doing the search knows what they are looking for, can try different solutions, and learn from multiple results to get to a working solution.

Why do you consider AI different? Nobody is expecting it always give correct solutions, just like nobody is expecting Googling something to always give the correct solution.

I’m not saying AI is useful, but I’m saying that a tool being fallible isn’t inherently a problem, whatever the tool is. As long as the user knows it’s fallible.