To me, it seems like they are arguing that “testing” whether a hammer can smash your thumb doesn’t actually provide any useful information on the safety of a hammer.

To me, it seems they are saying that Estwing makes a better hammer than Fischer-Price, even though the Fischer-Price hammer is far less likely to cause injury if you hit your thumb.

All this article says is that we shouldn’t stuff a general purpose LLM like ChatGPT into a Tickle-Me-Elmo.