How else could a remote signer sign on your behalf if the key is encrypted?