Yes, you have to trust someone.

My point is that cryptographic signatures can reduce that risk from having to trust tens of thousands of people to just one person.

That’s a hugely meaningful reduction of risk.