Why are brumbies a pest?

Brumbies are considered a pest because they 'destroy the environment'. Even though most of the damage is caused by humans living in those areas.