Is not at all an adverb?

Yes. The phrase "not at all" is an adverb modifying an adjective. It indicates that the characteristic, trait, or state is not present to a great degree.

Example:

The shop is not at all crowded today.

The manager was not at all happy with the changes.

The idiom "at all" and the Word "not" are both adverbs by themselves.