What silence refers to in sound of silence?

The "Sound of Silence" is a protest-style song about people who refuse to make their voices heard on things that matter (or at least matter to them).