Do you capitalize the phrase southeast?

Yes, "southeast" should be capitalized when referring to a specific geographic region (e.g., Southeast Asia) or as part of a proper noun (e.g., Southeast High School).