Do all queen ants have wings?

No. The average ant can't. Worker ants are wingless.

However, both male ants and the virgin queen ants have wings that allow hem to fly for the mating. Once fertilized, the queen ant bites off her wings and starts the colony. The male ants, having done their duty, die.

So the answer becomes - most can't, but some can and do in the early summer.