What part of speech is the word dozen?

It can be either a noun or adjective.

When used by itself, it means 12 of some item (the object is implicit)

Noun: I needed 5 eggs, so I bought a dozen.

When used before a noun, it is a numeral adjective (meaning 12 of some item or thing).

Adjective: A dozen cats were sitting outside my door.