It is largely an anti-tampering measure. Without it you could have things injected into the system. For example, a stalker could install a hidden tracking program as a service and then return your phone without you knowing.

Iirc it’s also a prerequisite for full-disk encryption on modern android. So, without it your user data is available to be dumped in an unencrypted state. Most phone thieves are interested in reselling the phone, so they’re provably not going to go through the effort and risk damage to the phone just to dump encrypted data from the chips directly. However, if it’s just available unencrypted from fastboot why not dump it? They could get info that could be used to blackmail or scam you or people you know. Or they could just sell the data.