I honestly think that the system you just described is close (in spirit at least) to the original American idea. The entire concept was a federal government that protected the borders and handled disputes between states. 

Everything else was “left to the states.” Granted, people living in those states voted and enacted laws that applied to everyone, but governments handled much less, meddled much less, and the culture and values were much more homogeneous (along with population being much smaller). 

It puts me in mind of a point from Montesquieu:

“It is in the nature of a republic to have only a small territory; otherwise, it can scarcely continue to exist. In a large republic, there are large fortunes, and consequently little moderation in spirits: the depositories are too large to put in the hands of a citizen; interests become particularised; at first a man feels he can be happy, great, and glorious without his homeland; and soon, that he can be great only on the ruins of his homeland.

In a large republic, the common good is sacrificed to a thousand considerations; it is subordinated to exceptions; it depends upon accidents. In a small one, the public good is better felt, better known, lies nearer to each citizen; abuses are less extensive there and consequently less protected.”

I am also put in mind of Franklin’s first fire department, and library. 

I certainly don’t disagree on the need to reduce and decentralize government - I think my difference with anarchists then is an issue of scale. I don’t believe man left entirely to his own devices will live peacefully (indeed I believe we would devolve into tribal warfare, and a might makes right attitude) but I agree that the current state of government is both too large to represent the best interests of any citizen, and far to powerful to prevent becoming abusive to the natural rights and freedoms of humankind. 

Thank you for the education on some points on anarchism. I feel less put off by the word now, and see some common ground.