Not always. Sometimes it is other people's egos.
If you're worried about other people's ego then you're worried about your own
Tell that to the guy who has a knife to your throat. Explain to him that it's really your ego in this case, causing the problem.