What does it mean when a British person says 'C of E'?

C of E is the British slang for the Church of England.

For example 'the Queen is head of the C of E'.