Is it a fire hazard when the wall outlet that has an electric heater plugged into it is hot?

Yes and Very Possible! Bad connections cause heat so the outlet may be defectice. The outlet or the supply line may also not be rated to supply the current (amperage) required by the heater.