What is an old male goat called?

It is called a buck. A castrated male goat is called a wether. A full grown female goat is called a doe. Babies are referred to as kids, in general, but specifically, bucklings and doelings.

Occasionally you will hear goats referred to as "nannies" or "billies." A nanny goat is another name for a doe, and a billy goat is another name for a buck. But those are not the proper terms.