Nip46 is far from perfect indeed.

To your points:
1. Nip46 server can be built into a relay, so you would be "just contacting signing server directly" to save one roundtrip. Not sure there is any practical difference btw HTTP and WS, aside from less experience with WS for most devs. The most important stuff you lose without a relay is:
- you can't run your nip46 server behind NAT without a relay in the middle, so even if you had some server you owned (Umbrel etc) it would require a complex port forwarding etc setup to host keys on it, and you'd expose your server to the wide internet and then hope it's not getting hacked, etc.
- you can't build a client-side solution like nsec.app where you don't need to own a server to self-host your nip46 signer
Essentially, adding a relay REMOVES a lot of complexity for practical self-hosted use and adoption. And if you wanted to save on round-trips, there will be nip46 signers built into relays.

2. Nip04 use sucks, we will need to upgrade at some point. Do you think we should start asap?