How much should it cost to replace a water pump?

That depends very much on the engine and the vehicle. Some can be done in less than an hour and some take 3-4 hours.