Yes but if we care about genetic diversity we should focus on Africa because it’s where almost all genetic diversity lies.  

I’m not saying we have too many people. More it’s ok if the population slowly declines. And humanity would be just fine with half or a quarter of the current population. We shouldn’t be worried that people are having less kids.