Since we first got easy access to various LLMs, I’ve been doing the opposite, asking obscure questions I know the answer to, trying to get a better understanding of what various models are really (not) capable of, and what data they’re (not) trained on, but it seems that you’re right and I’m in a minority. Most people treat the only LLM they know of as an oracle, and don’t seem to understand that it can write with confidence and still be incorrect. I’ve seen countless examples of just that, some funnier than other, so to me it has always been very obvious. It’s possible that using GPT-2 (back in the talktotransformer days), which was not configured for chat-style conversation but rather just to generate a continuation to the user’s input text, has actually helped me understand LLMs better and avoid using them in that common naive way, but I’m not sure how to make it just as clear to everyone else…