Under anarchy, groups could arise that see no problem with forcing others to do what they want, and such groups may become dominant. This is not unlikely. In fact, isn't this the definition of the State? Isn't this what we see in the USA right now? Isn't the current State nothing more than a dominant group lording it's power over others?
Is this really any different than an anarchical system gone awry--a society in which one group has gained superiority of force against another and opted to use that force to control them? Isn't this evidence against the present viability of anarchy?
As Robert Murhpy pointed out, anarchy could easily degenerate into warfare if the people in general see no problem with aggression. And as As John Milton wrote: "None can love freedom heartily, but good men; the rest love not freedom, but license." And Sam Adams would have agreed: "A general dissolution of the principles and manners will more surely overthrow the liberties of America than the whole force of the common enemy... While the people are virtuous they cannot be subdued; but once they lose their virtue, they will be ready to surrender their liberties to the first external or internal invader... " And I might add, they may take the liberty of others. It seems that this is the norm. States are prevalent. Maybe this is because of the scarcity of "virtue" among men. Men more often love comfort and ease than liberty and responsibility.
The viability of anarchy hangs on the character and beliefs of the people. Right now, I can't see it being sustainable. This does not mean that anarchy is not desirable. Nor does it mean that the State is legitimate. It simply means that anarchy is not possible until the minds of people change.