Is doesn't an adverb

No, it isn't. The Word doesn't is a contraction of the verb form "does not" where does is a helper verb and not is an adverb of negation.