Why do the boys refuse to vote for Jack as chief but slip off to join him later?

The boys are asked by Jack to vote against Ralph but it is only too plain to them that Ralph has done nothing wrong and that Jack simply wants to be leader, even to the point of lying about Ralph. The boys are embarrassed by Jack's behaviour and refuse to be party to his obvious attempt to overthrow Ralph in favour of himself. Later however the boys join Jack for the fun of hunting in preference to the rules and work involved in building shelters and maintaining the fire, which they feel is all that Ralph has to offer.