What causes caves to form?

the flow of water

+++

More specifically, in limestone as the vast majority of caves are, by the water dissolving the calcium carbonate that is the rock's primary constituent. The water starts as rain-water acidified slightly by absorbing atmospheric CO2, and this acid is what is really responsible.