Do all babies start out male?

Yes, all human embryos initially develop with a common set of structures that can lead to either male or female characteristics. This early stage is often referred to as having a "bipotential" state, where the gonads are neither male nor female. Around the sixth week of gestation, the presence of the Y chromosome triggers the development of male characteristics, while the absence of this chromosome leads to female development. Thus, while all embryos start with the potential to develop as either sex, genetic factors ultimately determine their sex.