If you hold the number it can’t get recycled into distribution. Signal does fall back to MFA codes over SMS from memory (I’ve recovered the signal account for my grandma, as I own the number), so anyone who controls the number controls the account.