If it’s not connected to the internet, the only way someone could reach it would be if they are already inside your network. If they are inside your network, either they are on one of your devices (and if it’s your computer, you can consider your keys compromised) or they have physical access and they can just take the drive. (Is the data on the drive encrypted?)