Our "intelligence" agencies already kill innocent people based entirely on metadata — because they simply live or work around areas that known terrorists occupy — now imagine if an AI was calling the shots.

So by your own scenario, intelligence agencies are already getting stuff wrong and making bad decisions using existing methodologies.

Why do you assume that new methodologies that involve LLMs will be worse at that? Why could they not be better? Presumably they're going to be evaluating their results when deciding whether to make extensive use of them.

"Mathematical magic tricks" can turn out to be extremely useful. That phrase can be used to describe all manner of existing techniques that are undeniably foundational to civilization.