What does it mean to hold your head higher than you heart?

It means that follow what you think before what you feel. Usually its the other way around though, like the saying "follow your heart", but in this case it means the opposite, to care more about something logically rather than having your feelings get involved.