What is the meaning of 'Don't throw the baby out with the bath water'?

This is a literal phrase, with neither symbolism nor great historical referent other than the pre-indoor Plumbing era. The phrase enjoins the baby washer not to lose the object of the exercise through misguided zeal or haste.