If I download e-mails from my account using a client like Thunderbird, can I just somehow copy them to a thumb drive and open them in whatever program later, without logging in to the account etc?

I want to delete my account, but need access to all my e-mails off.