Do you remove a walking air cast at night?

When I broke the growth plate in my foot my doctor told me to just sleep in the sock and take the boot off at night while sleeping.