I get the mathematical or engineering approach, but when it's literally the only thing in the way of freedom that changes things in my mind. 

You would have to save significantly more money according to the 4% rule to cover the mortgage than if you just pay it off and grow the gap between incomes and expenses. 

What you say is true, assuming inflation is what they claim it to be and wages keep up with inflation.