I agree with you, the government is no moral right to rule over you; even if the majority agree to form a representative government. No matter how a government is formed, it only exists as long as the majority continue to believe it is legitimate. However, the utopic vision of anarcho-capitalist ostracism, DROs and defense insurance providers also requires the majority concede to the immorality of government by force of the majority. 

 I will strongly suggest to watch and pay attention to the details the short series by Larken Rose - The island:
Part one:
https://www.youtube.com/watch?v=2pseCI0AxgM

Part two:
https://www.youtube.com/watch?v=3aSG0_trfp8

Part three:
https://www.youtube.com/watch?v=Qs0wPTutJHQ