I think that is a naive way of thinking it. 

My grandma was scared of black people and while that does indeed talk about how much of a fool she was, it didn't change the fact that Se was less likely to help black people in need than white people 

Or my uncle who was an insurance lawyer and believes that women are genetically worse at driving.

Or a friend who said that if he had a business he wouldn't hide women because they can get pregnant to get paid and do nothing

The point is that while words are just words and you can laugh it off, actions cannot