Are all bees female

Not quite. The queen and all worker bees are female. In summer, in each hive there will be somewhere between 200 and 500 males which are called drones.

In winter there will be no drones.