How and where do most caverns form?

By dissolution of limestone by ground-water in uplands where the rock's structural properties and hydrology favour cave development.