Yep, instead of a single address you should be able to issue keys that let people message you, and when you receive a message you should be able to see what key was used to send it.

And of course you should be able to revoke keys (tell your mail server to no longer accept messages signed with it).