The understanding of those passages that I've heard runs along a few parallel tracks.

The first is that God commands this to protect the Israelites.  As it turns out later, the Israelites don't completely uproot the inhabitants of the land, and they fall into the pagan practices of the Canaanites.  If they had obeyed God's commands, they would not have been as tempted towards idolatry.

A second reading is more metaphorical.  We can see in the Canaanites an image of sin and temptations.  If we are to be holy as God wishes for us, we must completely uproot sin and all of its occasions from our lives.  In this reading, God's command is directed at us today: we must purge all that separates us from him.

All of this is not necessarily incompatible with God's mercy.  The Canaanites were utterly lost in multifarious sins and idolatries.  Even the children would just grow up to heap condemnation upon themselves.  If they die at the hands of the Israelites that spares them from the future sins they would otherwise commit.