Is don't an adjective or an adverb?

It is a contraction, not a modifier, but it includes an adverb.

Don't means do not, where "do" is a helping verb, and "not" is an adverb.

The Word "don't" can also be a noun, meaning something that you should not do.