I don't see how it's any different from a password manager. You don't give your nsec to every service, you just use it to authenticate your identity. Yes if you get your master key compromised you are pwned but that's literally the same as having your password manager vault compromised.