What is it called when a cave has eroded all the way through?

Through... what?

It's still a cave, unless so short relative to its diameter that it can be called a Rock Arch.

There are caves that pass right through hills, sometimes because the land surface has been eroded down across the passage (common in the tropical karst of SE Asia).

In places a riverhas cut a cave down below its a valley floor then popped out again somewhere down-valley, leaving the intervening part of of its valley high and dry. I can think of quite a number here in Britain.

Through-trip caves are quite common in some areas, where a system has several entrances.

In fact the water has eroded its way all the way to the rising even if the lower reaches are not humanly-enterable in size.