Where do caterpillars go when it rains?

It depends on the type of butterfly.

Monarch Butterflies migrate from Canada and the United States to Mexico, Central America, and northern South America for the winter (they leave when summer ends), laying eggs on the way, the eggs then hatching and joining the other monarchs on the way. They hibernate through the winter. They wake up in the spring, which is when many of them lay eggs and die, to have the eggs continue the journey back up North.

Research some other butterflies or butterflies in general if this didn't completely answer your question.