With ssh, over 90% of the vulnerabilities are abusing the password mechanism. If you setup pre-shared keys, you are preventing the most common abuses, including in the realm of zero days.