Why do you bother washing bath towels?

To get rid of any residual dirt and germs and the particles of dead skin rubbed off by the towel.

Even when you have had a bath, you are not utterly, 100% clean. Bathing, like most processes, is imperfect. So a bath towel does not just remove water from your skin, it also picks up some small amount of dead skin or other residue. Eventually it will get dirty. Even so, I will grant that some people wash their bath towels more frequently than is really necessary. For some people it is just a form of ritualistic or obsessive cleanliness.