Why connect positive end of battery with positive end of voltmeter?

The leads of the voltmeter are labeled that way to get a positive readout when you connect the positive lead to the positive terminal of the battery (and negative to negative). Digital Voltmeters will read negative voltage if connected backwards. Voltmeters with a needle, would have the current flowing in such a way that the needle would try to deflect 'below zero' because the needle windings are designed for the current to flow in one direction, to cause positive deflection of the needle.