There is no contradiction because engagement goes up with follower count.

There is no algo here, and most people don't boost, so if someone isn't following you, they literally cannot see anything you write.