It’s a nice idea and a great flow, but it suffers from some  problems:

Low entropy
You are still giving your nsec to a million apps (here only one party has it)
No possibility of “password recovery” here recovery can be achieved 

 Yeah, there's no question your setup is far more elegant. This was just super easy to put together and test out. Definitely plan to move over to something more along the lines of what you're describing.