Even if you only publish to a private relay, anyone with access to that relay could broadcast your notes to any other relay
Hmmm you would need a way to communicate which notes are from the private relay and shouldn’t be reposted. Then have to trust those you invite not to re-note. Communicating privately might just continue to be the easier way.
Private chat groups definitely a better solution