Some clients will show you in the note settings, to which relays your note was broadcasted. That could be a hint 
Some clients, under the relay settings, will show some relays always with an error status. Could be a hint but the relay could also be in maintenance. 
Best way is still trial and error: remove one relay (one by one) and see if your experience is better.