It's because the thinner atmosphere at the higher altitude holds less heat. The atmosphere and its greenhouse effect is the reason why the night side of the Earth is not frozen, because the atmosphere holds infrared radiation given off by the Earth after it is irradiated by the sun. But at altitudes where the atmosphere is thinner, it is able to hold less of that warmth.