Turn that around. How much readers wouldn't dare interact with an account with a large following? Maybe afraid to say something 'wrong'. Or think their interactions are not worth it or will be snowed under in the replies? Would they participate more if these stats were hidden? Maybe...
I don't see growing a follower count as 'progress'. I personally never look at it and don't care.