Imagine if the IRS required you to give them your xpub. That's what I meant more than accidentally leaking it.