

and I’d need to have password access enabled in order to add the keys
Besides the other points, you could just add the public keys directly in the .ssh/authorized_keys(2) file of the server as long as you still have access from another device. That way you don’t have to enable passwords.























For that scenario you could also consider using certificate based login. Just store your root certificate in a safe place (like a Keepass) an then sign new keys for your new devices when you get one.
https://docs.ssh.com/manuals/server-admin/44/User_Authentication_with_Certificates.html