What is the air temperature like in the middle of the Atlantic Ocean?

It depends on the latitude, but you could look up the area you are interested in either on the NOAA buoys:
http://www.ndbc.noaa.gov/
Or maybe on this Navy map:
https://oceanography.navy.mil/legacy/web/LIBRARY/Metoc/Atlantic/Regional+NATL/SATANAL/OFA/Color+Composite/index.html