Literally, you’re right - in Latin it means “not following”. But in conventional usage, non-sequitur is more for things that are so completely out of place for the conversation.

Not a non-sequitor: "Okay, so based on this finding, ".

Non-sequitor: “Okay, so that’s great, but Michigan beating Ohio State means this is irrelevant”.