For taxes: initially it was because business owners were renouncing their citizenship as a way to avoid income and capital gains taxes, as well as estate taxes.

For the draft: during the Vietnam War people would flee to other countries (usually Canada) and renounce their US citizenship, even if they didn’t obtain citizenship elsewhere. Becoming a stateless person was (correctly) seen as less of a death sentence than being sent to Vietnam.