Does killing a bee attract other bees?

Yes it can do. There is a pheromone (perfume) present in the bee stinger that will attract other bees and cause them to attack.