What in our digestive system can become infected and have to be removed and has no purpose?

You are thinking of the appendix but it does have a purpose or we would have lost it long ago. The appendix is usually located in the right lower quadrant of the abdomen, near the right hip bone.
Its' position within the abdomen corresponds to a point on the surface known as McBurney's point . It has an immunologic function like the spleen.