Why cant hot-air balloons go to higher layers?

Hot air balloons rely on the temperature difference between the air inside and outside the balloon to generate lift. As you ascend, the temperature of the air outside the balloon decreases, reducing the lift. At higher altitudes, the balloon would need a larger temperature difference to stay afloat, which is difficult to achieve with a burner.