If your appendix bursts why would they not remove the appendix?

The usual first step is to drain away any residue from the ruptured appendix, and treat with antibiotics to prevent infection. Then they will remove it.

They would remove it right away if it had not yet burst.