Do you put a exclamation mark after an interjection?

It is used in most cases. There are seldom going to be a time when you wouldn't use one.