Yes, that is the hard part. But it can be done. Geoip blocking like only allow your country - blocking every China or Russian user etc. If you are selfhosting home and worry about your SSH access, then you can do a lot of things to block then early. It is all about authentication. Lets say you require VPN access in - example Wireguard. You could also do “port knocking” but that is not encrypted. You could require the user first has to be authenticated somewhere else, like require first Microsoft login and only then your ip is allowed.