I think this is one of the biggest hurdles for small government advocates to overcome, the "need" to have a government to provide an army. Here is what I think will happen, and why you will not need to worry about having a government army:
My prediction, is that enough liberty minded people will move to New Hampshire. Eventually, these people will stop obeying the local governments, mainly by not paying taxes. The local governments will harass the people (throw them in jail). But after a while, they will lose the fight and have to let them out of jail and let them not pay taxes. At this point, everyone else will stop paying local taxes and the local governments will just go away. Then we will see those people stop obeying the state government. The same thing will happen, and eventually the state government will lose and have to let them not pay taxes. As such, everyone in the state boundaries will stop paying taxes and the state government will just go away. Then the people living in New Hampshire will stop obeying the U.S. government. The U.S. government will try to make them pay, will realize the futility and let them live without obeying the U.S. government. The rest of the nation will see this and stop obeying as well, and you will see the U.S. government just, poof, disappear. The rest of the world will see this, and stop obeying the rest of the governments around the world. And then poof, all the governments on the earth will just disappear. As such, there will be no need for an army, because there will be no "people" to defend against, just other individuals. We will have a world of people instead of ethnic groups.
Now, you may disagree with whether or not this strategy will work. But that is not the point of the thread. The point is, that if it does work, then this will be the result.
At most, I think only 5% of the adult population would need to stop cooperating to have real change.
I like.
Another thing to consider is that defending against individuals is very very cheap. Every family just buys an AK47 and a thousand rounds of ammo. That sets you back for only $750 roughly. Add to that a little shooting-practice and you're under $100/month per family. And add a simple text messaging network in case a few others need to show up for support (in case some bad bikers or whatever are around).
To defend against armies, all you need on top of a heavily armed population are some anti-air/anti-tank missiles. So add $10K for every 400 persons (=$25 per person).
Now compare that (~ $50 per person a month) to your income tax and inflation tax (WHICH DOESN'T EVEN DEFEND YOU).
And I haven't even mentioned the different opportunity costs of crime versus work in a free society, or the effect of private rating and arbitration systems, making crime completely moot.