What is the rarest horse breed?

Probably the Caspian - was used by the Persians as chariot horses and believed to be extinct for a millenia until it was rediscovered in 1965 and there are only a handful in breeding programs to try to bring them back.

I'd say the Prezwalksi's Horse-There are roughly 1,500 of them left in captivity and 400 in the wild. All of them are the descendants of the same 14 founder horses.