You implied it.

You have a point that the very existence of government is an infringement. Unfortunately, it's not practical to ignore it completely. E.g. not use any roads.