What is the antonym of electorate?

There's no exact opposite for electorate, but the electorate is a body politic of registered voters, so at a stretch you could use 'non-voters' as an antonym.