Is post office a proper noun?

No, the compound, plural noun 'postage stamps' is a common noun, a general Word for any postage stamps of any kind.

A proper noun is the name of a specific person, place, or thing; for example, the Hawaiian Missionaries of 1851, or the Perot Postmaster's Provisionals of Bermuda.