When the balloon was in the sky will the air inside the balloon cool down if the flame was turned off?

Yes, if the flame is turned off, the air inside the balloon will gradually cool down. As the balloon rises, the surrounding air temperature decreases, and without the heat from the flame, the warmer air inside the balloon will lose heat to its cooler surroundings. This cooling effect will cause the balloon to become less buoyant over time.