What factor most determines how hot or cold something else feels when you touch it?

The factor that most determines how hot or cold something feels when you touch it is the temperature difference between the object and your skin. Heat will transfer from the warmer object to the cooler object, resulting in a sensation of hot or cold depending on the direction of heat flow.