Generally speaking it will be fine. SSH will also refuse keys with open permissions so you would notice if it was wide-open to other users of the device.
But you know if you are running random code or AI harnesses as that user it can be at risk. Or if you copy around the key all over the place it is more likely to leak. But generally speaking you are secure by default, just don’t do something dumb with the key and you’ll have no problems.
You should be good with caveats:
~/.ssh: chmod 700 ~/.sshchmod 600 ~/.ssh/id_rsaor id_ed25519, etc.chmod 644 ~/.ssh/id_rsa.pubneedanke@feddit.org 4 days ago
And of course encrypting the drive the files are on.