When measuring resistance should there be a voltage applied to the resistor?

No, the meter used to make the measurement will be applying whatever voltage to the resistor it needs to make the measurement. Any additional voltage will disturb this, resulting in at least an incorrect reading and at worst damaging or destroying the meter.