Not quite. It is like casual alcoholic saying, "oh I don't get drunk" until his liver starts complaining and either motivates him to get straight, or he is a toast.
With milk, you "tolerate" it while only wearing man-boobs, and same on women but less visible due to unhealthy aesthetic standards for millenniums, until you get cancer, blame tobacco or your wonderful genes that tolerates milk, and usually toast yourself with chemo.