When I did it I just forwarded my mails to the new service (fastmail) and changed mail address whenever a mail got forwarded in. When I didn’t receive a forwarded mail for a year or so I deleted my old mail and never looked back.

If you use a password manager this get easier though, since you can just lookup where the old mail is used.