Would it be best to leave the bees alone?

Bees, especilly bumblebees are generally harmless to humans. Furthermore no wasps or hornets will build a nest in the vicinity of a bees nest. Therefore, unless you have someone who is alergic to bee stings I would recomend just leaving the bees alone.