How drastic can you trim down an azelea bush?

That depends, but the safest time is immediately after it flowers. The harder you cut and the thicker and older the wood you cut into, the longer recovery will take. This is why it must be done fairly early in the season.