Is Popsicle a noun

Yes, the Word 'Popsicle' is a noun, a Word for a thing.

The noun 'Popsicle' is a proper noun, the name of a specific product, a registered trademark of Unilever.