Yes I think leaking the conversation key would expose the binding between the seal and rumor, not provably, but it would be highly unlikely to find a different conversation key that decrypted into a sensible rumor so effectively.  And the conversation key is post-HKDF so it doesn't expose your nsec.  This would have been great feedback for NIP-44 development.