What is it called when you explore caves?

Caving - as simple as that.

If the trip is genuine exploration, i.e. by the cave's discoverers as they find it, then we will use the Word "explore", but it is used loosely as after all, if you have never previously visited the cave it is your personal exploration of it.

It is NEVER called "spelunking" - that's a US-only Word that was coined legitimately by a group of cavers for themselves, but later became a slang term of derision aimed at novices and dilettantes!