Does adding white people ever increase diversity?