user A gets secret A and user B gets secret B and they can't communicate
it's a 2x2 grid, two cases where A and B pubkeys are 3s you can't get the same shared secret
most generators already either reject 3s or subtract the curve order from the secret to get the opposite sign, but even with this in mind you still have teh problem that you have to try a second one to test if it's just the wrong one and that only gives you 25% of cases where they are wrong, in the other direction, where you guess the pubkey prefix/sign, it works
i have seen this in DMs before and it was infuriating, but i didn't test it enough, i'd generate a key, have problems, and without knowing what was going on, that key would be discarded and another one generated, and 50% of the time it would work and seem like the stars were out of alignment, but it's a bug in the protocol that needs to be strongly stipulated about the keys being ONLY 2 prefix, even pubkeys because of ECDH
the ways around it are difficult, the only solution i see is signalling optionally in kind 0 if you have an odd pubkey
that solves the problem
if two people have 3 keys and their software guesses 2s both sides can't communicate, otherwise, you have a case where one way works the other doesn't