When the OD OFF indicator is lit on the dash does that mean I'm in Overdrive or I need to press it to turn it off?

If the light says off then your overdrive is turned off, push the button again and the light will turn off and overdrive will be on again.