You *can* verify the entire ledger yourself, but *do* you? 

Your argument is akin to saying "I'm not fat because I *can* lift weights (but don't)"

Being able to do something, but never doing it, is functionally indistinguishable. Otherwise you're just trusting others.

The few that do actually audit are able to truly say that, but passive Bitcoin node runners have no/much weaker arguments against Monero. Most Bitcoiners don't even run a node let alone manually verify.