Do you have to prime all siding before putting paint on it?

No. Do not prime any wood if you are going to stain it. Stains penetrate wood fibers, while primer and paint remain on the surface of the wood. If you put a primer on first, it will cover the grain and the stain won't work.