Why H-G men are the antagonists in the story harrison bergeron?

Society was the antagonist for letting things get out of hand. HG men and Diana Moon were just a few faces to a billion headed monster.